HomePhabricator Dev

Bug 1467359 - Implement fetch-url jobs using a transform; r=tomprince

This commit no longer exists in the repository. It may have been part of a branch which was deleted.

Description

Bug 1467359 - Implement fetch-url jobs using a transform; r=tomprince

Previously, the fetch kind was defined by a job "using" flavor.

An upcoming commit will need to derive new tasks from fetch job
definitions. In order to do this, we require a transform. And
this transform would need access to the original data from the
job description. But by the time the "using" transform runs, a
lot of this data is thrown away. It would be possible to stuff
the meaningful metadata inside attributes on the created task.
But this would result in the fetch logic being fragmented across
multiple Python modules (a fetch-specific transform and a job/using-
specific transform). I think it is better to keep all that logic
in a single Python module.

This commit converts the "using: fetch-url" job transform to a
dedicated transform for the fetch kind.

Since we're now using a dedicated transform, we no longer need
to use the normal job schema for defining fetch jobs. So we
refactor the schema a little so it is simpler.

I verified the taskgraph output is nearly identical to before by
diffing the JSON output of mach taskgraph full. Aside from the
additions of ['worker']['implementation'] and ['worker']['os']
fields (which seem to be required by a schema somewhere), everything
was identical.

Differential Revision: https://phabricator.services.mozilla.com/D1574

Details

Provenance
• gpsAuthored on

Commit No Longer Exists

This commit no longer exists in the repository.