Deep Learning from MAGIC Raw Data
Jarred Green
MAGIC SOBO Meeting — 30 June 2025
Max Planck Institute for Physics
jgreen@mpp.mpg.de
Outline
- Current Reconstruction Method
- Graphs
- Current Methods
- ICRC and Next Steps
Jarred Green - jgreen@mpp.mpg.de
Jarred Green - jgreen@mpp.mpg.de
A way to include as much information as possible?
1. Current Reconstruction
- Particle Type
- Energy
- Arrival Direction
Jarred Green - jgreen@mpp.mpg.de
Proton
Gamma
Muon
Find a new ways to represent raw data
1. Current Reconstruction
2. Graphs
Jarred Green - jgreen@mpp.mpg.de
Jarred Green - jgreen@mpp.mpg.de
Lets go from images
to graphs
Credit: J. Leskovec (Stanford CS224)
for MAGIC?
GNNs can effectively deal with
- irregular camera geometry
- sparse data
- non gridlike pixels
- arrays with different cameras (ie CTA)
- information on each node (pixel) and edge (connection)
Jarred Green - jgreen@mpp.mpg.de
2. Graphs
Jarred Green - jgreen@mpp.mpg.de
time
- Same event with a simple threshold in p.e.
2. Graphs
Jarred Green - jgreen@mpp.mpg.de
time
Real data event
Can we 'steal' this timing information from OFF data?
YES!
"time calibration pasting"
2. Graphs
Jarred Green - jgreen@mpp.mpg.de
Can we 'steal' this timing information from OFF data?
YES!
"time calibration pasting"
- ✅ Contains all signal and background information
- ❌ Huge graphs with 100K+ nodes
- ❌ Training ~days
2. Graphs
Jarred Green - jgreen@mpp.mpg.de
time
Can we make these graphs smaller somehow?
cleaning!
- ✅ Small graphs with 200-2K nodes
- ✅ Training ~hours
- ❌ Maybe loosing some signal information
2. Graphs
an IceCube case study
- Created their own GNN framework for Neutrino detection
- 15-20% improvement in reconstruction of energy, zenith, direction at low energies
- Can do realtime reconstruction onsite with 1 GPU
Credit: R. Abbasi et al. [arXiv:2209.03042]
Jarred Green - jgreen@mpp.mpg.de
2. Graphs
Jarred Green - jgreen@mpp.mpg.de
Goal: get calibrated data with timing information
MARS - sorcerer
Python package magic-gnn
.parquet or LMDB files exported, about ~1GB / min of data
stereo event matching
Transformers with
cleaned graphs
GNNs on cleaned and uncleaned graphs
Calibration
MExportCSVMExportParquetdata standardization
time calibration pasting
Python package magic-gnn
✅✅2️⃣
1️⃣
3. Current Methods
Jarred Green - jgreen@mpp.mpg.de
Training Data: 150K MC events, za05to35, ST0307
GNNs with cleaned graphs
Real Data: ST0307 Triangulum II (OFF)
- Training with GraphNet (IceCube)
- Dynedge architecture
- Telescopes talk with cross-attention
- Outputs 'gammaness'
3. Classification
1️⃣
Jarred Green - jgreen@mpp.mpg.de
Training Data: 2M gamma events, za05to35, ST0307
Transformers on
MAGIC graphs
Real Data: ST0307 Triangulum II (OFF)
- Similar idea to ChatGPT
- Pixels in time are treated like 'sentences'
- Keeping a smaller GNN built into the model
3. Direction
- Loss: Angular error plus FoV error
- No other feature extraction or physics information
2️⃣
Training now
- Running since ~12hrs at MPP
- Current 68% error at ~1º
4. What to show at ICRC
- Focus on methods only
- No need for performance plots
- Results will continue to improve over the next month(s)
- Save results for publication and TeVPA
- Preliminary Theta2 of Crab Nebula?
Jarred Green - jgreen@mpp.mpg.de
- Nudge direction reconstruction model towards DISP methods
- Provide with global features or tuned physics-informed models/losses
- Test on Crab Nebula sample
- Need drive reports?
- Try both architectures with Energy reconstruction
- Foundation model (all tasks together)
- Collaborate with IceCube and ML experts at TUM
4. Next Steps
Jarred Green - jgreen@mpp.mpg.de
Thank you!
Jarred Green - jgreen@mpp.mpg.de
Credit: G. Ceribella
Jarred Green - jgreen@mpp.mpg.de
time
- Sample MC gamma event
2. Graphs
Jarred Green - jgreen@mpp.mpg.de
Long-term ideas
- "Pedestal pasting"
- Semi-supervised warmup
- Unsupervised learning for classification
- MAGIC+LST combined analysis
- Work with ML experts at TUM
- ➡️ Investigate point cloud object detection
Jarred Green - jgreen@mpp.mpg.de
Long-term ideas
➡️ Investigate point cloud object detection
Interesting GNN model:
Point-GNN: Graph Neural Network for 3D Object Detection in a Point Cloud
ICRC SOBO GNNs
By astrojarred
