Rediscover the known Universe with NASA dataset

Rediscover the known Universe with NASA dataset Horacio Gonzalez @LostInBrittany Emmanuel Feller @moyowi

Horacio Gonzalez @LostInBrittany Spaniard lost in Brittany, developer, dreamer and all-around geek #HelloExoWorld @LostInBrittany @moyowi

Emmanuel Feller @moyowi Développeur passionné #HelloExoWorld @LostInBrittany @moyowi

Pierre & Aurélien we ♥ you #HelloExoWorld @LostInBrittany @moyowi

HelloExoWorld Looking for exoplanets in NASA datasets #HelloExoWorld @LostInBrittany @moyowi

HelloExoWorld Once upon a time… #HelloExoWorld @LostInBrittany @moyowi

What not to do if you love astronomy To live in Brest #HelloExoWorld @LostInBrittany @moyowi

Looking for solutions Computer stuff Astronomy Mixing passions #HelloExoWorld @LostInBrittany @moyowi

Google is your friend… Let’s find a project #HelloExoWorld @LostInBrittany @moyowi

Exoplanets? Planets orbiting stars far away #HelloExoWorld @LostInBrittany @moyowi

How do we find them? The transit method seems the best #HelloExoWorld @LostInBrittany @moyowi

Exoplanets detection From theory to practice #HelloExoWorld @LostInBrittany @moyowi

The transit method Credits: NASA’s Goddard Space Flight Center #HelloExoWorld @LostInBrittany @moyowi

How do we look for transits? Image credits : NASA Image credits : NASA Kepler Tess #HelloExoWorld @LostInBrittany @moyowi

#HelloExoWorld @LostInBrittany @moyowi

Watching the sky By Carter Roberts [Public domain], via Wikimedia Commons #HelloExoWorld @LostInBrittany @moyowi

Kepler image A star : 12*12px #HelloExoWorld @LostInBrittany @moyowi

And what kind of data we get? Pleiades By NASA, ESA, AURA/Caltech, Palomar Observatory. Via Wikimedia Common #HelloExoWorld @LostInBrittany @moyowi

Well, that’s the problem Seven stars, seven different profiles #HelloExoWorld @LostInBrittany @moyowi

Kinda big data Over 40 million light curves #HelloExoWorld @LostInBrittany @moyowi

Big AND open data Lots of datasets in #opendata #HelloExoWorld @LostInBrittany @moyowi

And we can help with that! Let’s use our tools to analyse the data #HelloExoWorld @LostInBrittany @moyowi

Time Series To analyse Kepler datasets #HelloExoWorld @LostInBrittany @moyowi

Kepler: spatial Time Series Definition of Time Series: A series of data points indexed in time order #HelloExoWorld @LostInBrittany @moyowi

Time Series ● ● ● ● ● ● ● Stock Market Analysis Economic Forecasting Budgetary Analysis Process and Quality Control Workload Projections Census Analysis … #HelloExoWorld @LostInBrittany @moyowi

Time Series Applications: ▪ Understanding the data ▪ Fit a model – – Monitoring Forecasting #HelloExoWorld @LostInBrittany @moyowi

Time Series Stock market Analytics Economic Forecasting $$$ Study & Research #HelloExoWorld @LostInBrittany @moyowi

Time Series Many specific analytical tools: ● Moving average ● ARMA (AutoRegressive Moving Average) ● Multivariate ARMA models ● ARCH (AutoRegressive Conditional Heteroscedasticity) ● Dynamic time warping ● … #HelloExoWorld @LostInBrittany @moyowi

Time Series Specific application of general tools ● Artificial neural networks ● Hidden Markov model ● Fourier & Wavelets transforms ● Entropy encoding ● … #HelloExoWorld @LostInBrittany @moyowi

Dealing with Time Series The 3 ‘v’ #HelloExoWorld @LostInBrittany @moyowi

Monitoring OVH with Time Series #HelloExoWorld @LostInBrittany @moyowi

OVH Metrics A metrics data platform #HelloExoWorld @LostInBrittany @moyowi

Tools to deal with Time Series Many options #HelloExoWorld @LostInBrittany @moyowi

Metrics Data Platform #HelloExoWorld @LostInBrittany @moyowi

Metrics’ metrics ● 1.5M datapoints/s, 24/7 ● Peaks at ~10M datapoints/s ● 500M unique series #HelloExoWorld @LostInBrittany @moyowi

Metrics Data Platform + + #HelloExoWorld @LostInBrittany @moyowi

Why Warp 10? Warp 10 is a software platform that ● Ingests and stores time series ● Manipulates and analyzes time series #HelloExoWorld @LostInBrittany @moyowi

Analytics is the key to success Fetching data is only the tip of the iceberg #HelloExoWorld @LostInBrittany @moyowi

Manipulating Time Series with Warp 10 A true Time Series analysis toolbox ● ● ● Hundreds of functions Manipulation frameworks Analysis workflow #HelloExoWorld @LostInBrittany @moyowi

Anatomy of a time series Each time series is composed of: org.nasa.kepler.starlight { keplerId: 52163778 } Metadata ▪ ▪ Class name Labels Datapoints ▪ ▪ Timestamp Value #HelloExoWorld @LostInBrittany @moyowi

Class names and labels Class names define the kind of measure ▪ Starlight, heart rate, speed… org.nasa.kepler.starlight { keplerId: 52163778 } Labels define particular traits of a TS ▪ Device Id, Device Type, Private User Id… #HelloExoWorld @LostInBrittany @moyowi

A match made in heaven Warp 10, OVH Metrics and HelloExoWorld #HelloExoWorld @LostInBrittany @moyowi

What we have done ● ● ● ● Downloaded and parsed 40 millions of FITS files Pushed it to OVH Metrics Select a cool subset as training set Verified we could find the same planets as NASA #HelloExoWorld @LostInBrittany @moyowi

From kepler-11 raw data #HelloExoWorld @LostInBrittany @moyowi

To (candidates) exoplanets #HelloExoWorld @LostInBrittany @moyowi

Your job #HelloExoWorld @LostInBrittany @moyowi

Let’s get started!

Connect to https://bit.ly/2H7Z5b3 2. Enjoy! #HelloExoWorld @LostInBrittany @moyowi

What’s next? Where do we go from here? #HelloExoWorld @LostInBrittany @moyowi

Only the beginning Better detection New import method Explorer Deep learning satellite/star location #HelloExoWorld Yours? @LostInBrittany @moyowi

A growing team #HelloExoWorld @LostInBrittany @moyowi

And you! Join us! https://helloexo.world https://xkcd.com/1371/ #HelloExoWorld @LostInBrittany @moyowi

OVH Platform Come speak with us about your kubernetes, time-series or observability projects and OVH Platform #HelloExoWorld @LostInBrittany @moyowi

Thank you, dear sponsors! #HelloExoWorld @LostInBrittany @moyowi

Thank you! #HelloExoWorld @LostInBrittany @moyowi