Open Bug 1990006 Opened 1 year ago Updated 3 months ago

Create a ETL workflow for the gecko-trace component

Categories

(Toolkit :: Gecko Trace, task, P2)

Product:
Toolkit
Component:
Gecko Trace
Type:
task

Tracking

()

People

(Reporter: michael, Assigned: michael)

References

(Blocks 1 open bug)

Details

(Keywords: leave-open)

User Story

The Gecko-Trace component can send error diagnostics to the Glean backend on Nightly builds when the `toolkit.gecko-trace.enable` preference is true.

However, these Glean messages aren’t directly accessible; an ETL workflow is required to transform them into a queryable, usable form. Without this step, the error-tracing system cannot be used as intended.

A patch for the workflow exists, but the ETL platform is owned by the Data Engineering team, and triggering its CI checks and deploying the workflow require permissions held by that team. Therefore, completion of this workflow depends on support from Data Engineering.

Attachments

(3 files)

[mozilla/bigquery-etl] Bug 1990006 - Create an initial ETL workflow for gecko-trace component (#8165)
49 bytes, text/x-github-pull-request
Details | Review
Bug 1990006 - Fix `send_in_pings` pointing to old 'traces' ping r=jari,mvanstraten
48 bytes, text/x-phabricator-request
Details | Review
[mozilla/bigquery-etl] Bug 1990006 - Create an initial ETL workflow for gecko-trace component (#9553)
49 bytes, text/x-github-pull-request
Details | Review
No description provided.
Assignee: nobody → mvanstraten
Blocks: 1950559
User Story: (updated)
Severity: -- → S3
Priority: -- → P2
Keywords: leave-open
Pushed by agoloman@mozilla.com: https://github.com/mozilla-firefox/firefox/commit/2dc05bedeff3 https://hg.mozilla.org/integration/autoland/rev/457cf84a902a Revert "Bug 1990006 - Fix `send_in_pings` pointing to old gecko-trace ping r=jari" for causing failures @pings_yamls.cached.stub.

Backed out for causing failures @pings_yamls.cached.stub.

Flags: needinfo?(michael)

I have a hunch why this cause a failure, re-running on try now. Will get back to you soon.

So apparently you can not use underscores in the send_in_ping names (very fun :)).

Flags: needinfo?(michael)

According to the failure log:
'gecko_trace' is not valid under any of the given schemas [task 2025-11-12T18:28:59.053+00:00] 18:28:59 INFO - 'gecko_trace' does not match '^[a-z][a-z0-9-]{0,29}$'
So, underscores are not allowed by the parser

However, https://github.com/mozilla/bigquery-etl/pull/8165/files - in this server side patch, underscores are used

So, the proper way is to fix a parser in order to allow underscores (which is not possible since this is a third party library)

Attachment #9526163 - Attachment description: Bug 1990006 - Fix `send_in_pings` pointing to old gecko-trace ping r=jari → Bug 1990006 - Fix `send_in_pings` pointing to old gecko-trace ping r=jari,mvanstraten
Attachment #9526163 - Attachment description: Bug 1990006 - Fix `send_in_pings` pointing to old gecko-trace ping r=jari,mvanstraten → Bug 1990006 - Fix `send_in_pings` pointing to old 'traces' ping r=jari,mvanstraten
Pushed by abutkovits@mozilla.com: https://github.com/mozilla-firefox/firefox/commit/925e7fb2de2a https://hg.mozilla.org/integration/autoland/rev/7fdf8e16e4ef Revert "Bug 1990006 - Fix `send_in_pings` pointing to old 'traces' ping r=jari" for causing failures at generated-metrics.yaml.
Flags: needinfo?(michael) → needinfo?(aiunusov)
Attachment #9526163 - Attachment description: Bug 1990006 - Fix `send_in_pings` pointing to old 'traces' ping r=jari,mvanstraten → WIP: Bug 1990006 - Fix `send_in_pings` pointing to old 'traces' ping r=jari,mvanstraten
Attachment #9526163 - Attachment description: WIP: Bug 1990006 - Fix `send_in_pings` pointing to old 'traces' ping r=jari,mvanstraten → Bug 1990006 - Fix `send_in_pings` pointing to old 'traces' ping r=jari,mvanstraten

patch needs another review :)
https://phabricator.services.mozilla.com/D272244

Flags: needinfo?(aiunusov) → needinfo?(jjalkanen)
Flags: needinfo?(jjalkanen)
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: