Deep Learning with MAGIC Raw Data
Jarred Green
MAGIC Meeting Siena — 9 June 2025
Max Planck Institute for Physics
jgreen@mpp.mpg.de
Outline
- Current Reconstruction Method
- Graphs
- Current Methods
- What's next
Jarred Green - jgreen@mpp.mpg.de
Jarred Green - jgreen@mpp.mpg.de
1. Current Reconstruction
Raw Data
Images
{
'size': 546.8863769783503,
'xc': -3.7898364035228065,
'yc': 7.294274369102854,
'length': 7.767692666363586,
'width': 1.7648053936769377,
'delta': -0.5687518727314761,
...
}Parameters
- Particle Type
- Energy
- Arrival Direction
Jarred Green - jgreen@mpp.mpg.de
A way to include as much information as possible?
1. Current Reconstruction
Jarred Green - jgreen@mpp.mpg.de
Proton
Gamma
Muon
Find a new ways to represent raw data
1. Current Reconstruction
2. Graphs
Jarred Green - jgreen@mpp.mpg.de
Jarred Green - jgreen@mpp.mpg.de
Lets go from images
to graphs
Credit: J. Leskovec (Stanford CS224)
for MAGIC?
GNNs can effectively deal with
- irregular camera geometry
- sparse data
- non gridlike pixels
- arrays with different cameras (ie CTA)
- information on each node (pixel) and edge (connection)
Jarred Green - jgreen@mpp.mpg.de
2. Graphs
Jarred Green - jgreen@mpp.mpg.de
time
- Sample MC gamma event
2. Graphs
Jarred Green - jgreen@mpp.mpg.de
time
- Same event with a simple threshold in p.e.
2. Graphs
Jarred Green - jgreen@mpp.mpg.de
time
Real data event
Can we 'steal' this timing information from OFF data?
YES!
"time calibration pasting"
GNNs with
calibrated graphs
1️⃣
2. Graphs
Jarred Green - jgreen@mpp.mpg.de
Can we 'steal' this timing information from OFF data?
YES!
"time calibration pasting"
GNNs with
calibrated graphs
1️⃣
- ✅ Contains all signal and background information
- ❌ Huge graphs with 100K+ nodes
- ❌ Training ~days
2. Graphs
Jarred Green - jgreen@mpp.mpg.de
time
Can we make these graphs smaller somehow?
cleaning!
GNN with
cleaned graphs
2️⃣
- ✅ Small graphs with 200-2K nodes
- ✅ Training ~hours
- ❌ Maybe loosing some signal information
2. Graphs
an IceCube case study
- Created their own GNN framework for Neutrino detection
- 15-20% improvement in reconstruction of energy, zenith, direction at low energies
- Can do realtime reconstruction onsite with 1 GPU
Credit: R. Abbasi et al. [arXiv:2209.03042]
Jarred Green - jgreen@mpp.mpg.de
2. Graphs
Jarred Green - jgreen@mpp.mpg.de
Goal: get calibrated data with timing information
MARS - sorcerer
Python package magic-gnn
.parquet or LMDB files exported, about ~1GB / min of data
stereo event matching
GNN with
cleaned graphs
GNNs with
calibrated graphs
Calibration
MExportCSVMExportParquetdata standardization
time calibration pasting
Python package magic-gnn
3. Current Methods
Jarred Green - jgreen@mpp.mpg.de
Goal: get calibrated data with timing information
MARS - sorcerer
Python package magic-gnn
.parquet or LMDB files exported, about ~1GB / min of data
stereo event matching
GNN with
cleaned graphs
GNNs with
calibrated graphs
Calibration
MExportCSVMExportParquetdata standardization
time calibration pasting
Python package magic-gnn
✅✅2️⃣
1️⃣
3. Current Methods
Jarred Green - jgreen@mpp.mpg.de
Training Data: 150K MC events, za05to35, ST0307
GNN with
cleaned graphs
GNNs with
calibrated graphs
2️⃣
1️⃣
Real Data: ST0307 Crab Nebula and Triangulum II (OFF)
- Static Edges
- Graph convolutions
- Captures local neighborhoods around signal pixels
- Classification only for now
- Training with GraphNet (IceCube)
- Dynedge architecture
- All three tasks
3. Current Methods
- Check in with me later this week for more details and results!
- Create customized MC simulations
- Pedestal pasting
- Moon data
4. What's next
Join the Deep Learning group!
Thank you!
Jarred Green - jgreen@mpp.mpg.de
1. Current Reconstruction
Jarred Green - jgreen@mpp.mpg.de
1. Trigger + 2. Raw data
- charge measured in each pixel
- 50 time slices recorded in each pixel [waveforms]
Jarred Green - jgreen@mpp.mpg.de
3. Image cleaning
- isolate main part of image
- average pixel arrival time
1. Current Reconstruction
Jarred Green - jgreen@mpp.mpg.de
4. Image parameterization
{
'size': 546.8863769783503,
'xc': -3.7898364035228065,
'yc': 7.294274369102854,
'length': 7.767692666363586,
'width': 1.7648053936769377,
'delta': -0.5687518727314761,
...
}1. Current Reconstruction
"Hillas Parameters" (1977!)
{
'size': 546.8863769783503,
'xc': -3.7898364035228065,
'yc': 7.294274369102854,
'length': 7.767692666363586,
'width': 1.7648053936769377,
'delta': -0.5687518727314761,
...
}"Hillas Parameters"
Random Forest
The ML part
Jarred Green - jgreen@mpp.mpg.de
1. Current Reconstruction
{
'size': 546.8863769783503,
'xc': -3.7898364035228065,
'yc': 7.294274369102854,
'length': 7.767692666363586,
'width': 1.7648053936769377,
'delta': -0.5687518727314761,
...
}tl;dr
- Particle Type
- Energy
- Arrival Direction
ML
Jarred Green - jgreen@mpp.mpg.de
1. Current Reconstruction
Jarred Green - jgreen@mpp.mpg.de
Is there a deep learning model where we can keep as much information as possible from RAW data?
{
'size': 546.8863769,
'xc': -3.7898364035,
'yc': 7.294274369,
'length': 7.767692666,
'width': 1.7648053936,
'delta': -0.5687518727,
...
}2. Data pipeline
Jarred Green - jgreen@mpp.mpg.de
Long-term ideas
- Investigate unsupervised learning
- MAGIC+LST combined analysis
- ➡️ Look at IceCube Kaggle competition
- ➡️ Investigate point cloud object detection
Jarred Green - jgreen@mpp.mpg.de
Long-term ideas
➡️ Look at IceCube Kaggle competition
Jarred Green - jgreen@mpp.mpg.de
Long-term ideas
➡️ Investigate point cloud object detection
Interesting GNN model:
Point-GNN: Graph Neural Network for 3D Object Detection in a Point Cloud
MAGIC 2025 Siena - GNNs
By astrojarred
