Deep Learning with MAGIC Raw Data

Jarred Green

MAGIC Meeting Siena — 9 June 2025

Max Planck Institute for Physics

jgreen@mpp.mpg.de

Outline

  1. Current Reconstruction Method
  2. Graphs
  3. Current Methods
  4. What's next

Jarred Green - jgreen@mpp.mpg.de

Jarred Green - jgreen@mpp.mpg.de

1. Current Reconstruction

Raw Data

Images

{
	'size': 546.8863769783503,
	'xc': -3.7898364035228065,
	'yc': 7.294274369102854,
	'length': 7.767692666363586,
	'width': 1.7648053936769377,
	'delta': -0.5687518727314761,
	 ...
}

Parameters

  • Particle Type
  • Energy
  • Arrival Direction

Jarred Green - jgreen@mpp.mpg.de

A way to include as much information as possible?

1. Current Reconstruction

Jarred Green - jgreen@mpp.mpg.de

Proton

Gamma

Muon

Find a new ways to represent raw data

1. Current Reconstruction

2. Graphs

Jarred Green - jgreen@mpp.mpg.de

Jarred Green - jgreen@mpp.mpg.de

Lets go from images

to graphs

Credit: J. Leskovec (Stanford CS224)

for MAGIC?

GNNs can effectively deal with

  • irregular camera geometry
  • sparse data
  • non gridlike pixels
  • arrays with different cameras (ie CTA)
  • information on each node (pixel) and edge (connection)

Jarred Green - jgreen@mpp.mpg.de

2. Graphs

Jarred Green - jgreen@mpp.mpg.de

time

  • Sample MC gamma event

2. Graphs

Jarred Green - jgreen@mpp.mpg.de

time

  • Same event with a simple threshold in p.e.

2. Graphs

Jarred Green - jgreen@mpp.mpg.de

time

Real data event

Can we 'steal' this timing information from OFF data?
YES!

"time calibration pasting"

GNNs with
calibrated graphs

1️⃣

2. Graphs

Jarred Green - jgreen@mpp.mpg.de

Can we 'steal' this timing information from OFF data?
YES!

"time calibration pasting"

GNNs with
calibrated graphs

1️⃣
  • ✅ Contains all signal and background information
  • ❌ Huge graphs with 100K+ nodes
  • ❌ Training ~days

2. Graphs

Jarred Green - jgreen@mpp.mpg.de

time

Can we make these graphs smaller somehow?

cleaning!

GNN with
cleaned graphs

2️⃣

  • ✅ Small graphs with 200-2K nodes
  • ✅ Training ~hours
  • ❌ Maybe loosing some signal information

2. Graphs

an IceCube case study

  • Created their own GNN framework for Neutrino detection
  • 15-20% improvement in reconstruction of energy, zenith, direction at low energies
  • Can do realtime reconstruction onsite with 1 GPU

Credit: R. Abbasi et al. [arXiv:2209.03042]

Jarred Green - jgreen@mpp.mpg.de

2. Graphs

Jarred Green - jgreen@mpp.mpg.de

Goal: get calibrated data with timing information

MARS - sorcerer

Python package magic-gnn

.parquet or LMDB files exported, about ~1GB / min of data

stereo event matching

GNN with
cleaned graphs

GNNs with
calibrated graphs

Calibration

MExportCSV
MExportParquet

data standardization

time calibration pasting

Python package magic-gnn

3. Current Methods

Jarred Green - jgreen@mpp.mpg.de

Goal: get calibrated data with timing information

MARS - sorcerer

Python package magic-gnn

.parquet or LMDB files exported, about ~1GB / min of data

stereo event matching

GNN with
cleaned graphs

GNNs with
calibrated graphs

Calibration

MExportCSV
MExportParquet

data standardization

time calibration pasting

Python package magic-gnn
✅
✅

2️⃣

1️⃣

3. Current Methods

Jarred Green - jgreen@mpp.mpg.de

Training Data: 150K MC events, za05to35, ST0307

GNN with
cleaned graphs

GNNs with
calibrated graphs

2️⃣

1️⃣

Real Data: ST0307 Crab Nebula and Triangulum II (OFF)

  • Static Edges
  • Graph convolutions
  • Captures local neighborhoods around signal pixels
  • Classification only for now
  • Training with GraphNet (IceCube)
  • Dynedge architecture
  • All three tasks

3. Current Methods

  • Check in with me later this week for more details and results!
  • Create customized MC simulations
  • Pedestal pasting
  • Moon data

4. What's next

Join the Deep Learning group!

Thank you!

Jarred Green - jgreen@mpp.mpg.de

1. Current Reconstruction

Jarred Green - jgreen@mpp.mpg.de

1. Trigger + 2. Raw data

  • charge measured in each pixel
  • 50 time slices recorded in each pixel [waveforms]

Jarred Green - jgreen@mpp.mpg.de

3. Image cleaning

  • isolate main part of image
  • average pixel arrival time

1. Current Reconstruction

Jarred Green - jgreen@mpp.mpg.de

4. Image parameterization

{
	'size': 546.8863769783503,
	'xc': -3.7898364035228065,
	'yc': 7.294274369102854,
	'length': 7.767692666363586,
	'width': 1.7648053936769377,
	'delta': -0.5687518727314761,
	 ...
}

1. Current Reconstruction

"Hillas Parameters" (1977!)

{
	'size': 546.8863769783503,
	'xc': -3.7898364035228065,
	'yc': 7.294274369102854,
	'length': 7.767692666363586,
	'width': 1.7648053936769377,
	'delta': -0.5687518727314761,
	 ...
}

"Hillas Parameters"

Random Forest

The ML part

Jarred Green - jgreen@mpp.mpg.de

1. Current Reconstruction

{
	'size': 546.8863769783503,
	'xc': -3.7898364035228065,
	'yc': 7.294274369102854,
	'length': 7.767692666363586,
	'width': 1.7648053936769377,
	'delta': -0.5687518727314761,
	 ...
}

tl;dr

  • Particle Type
  • Energy
  • Arrival Direction

ML

Jarred Green - jgreen@mpp.mpg.de

1. Current Reconstruction

Jarred Green - jgreen@mpp.mpg.de

Is there a deep learning model where we can keep as much information as possible from RAW data?

{
	'size': 546.8863769,
	'xc': -3.7898364035,
	'yc': 7.294274369,
	'length': 7.767692666,
	'width': 1.7648053936,
	'delta': -0.5687518727,
	 ...
}

2. Data pipeline

Jarred Green - jgreen@mpp.mpg.de

Long-term ideas

  • Investigate unsupervised learning
  • MAGIC+LST combined analysis
  • ➡️ Look at IceCube Kaggle competition
  • ➡️ Investigate point cloud object detection

Jarred Green - jgreen@mpp.mpg.de

Long-term ideas

➡️ Look at IceCube Kaggle competition

Jarred Green - jgreen@mpp.mpg.de

Long-term ideas

➡️ Investigate point cloud object detection

Interesting GNN model:
Point-GNN: Graph Neural Network for 3D Object Detection in a Point Cloud