Main Page: Difference between revisions

From Global Meteor Network
Jump to navigation Jump to search
 
(64 intermediate revisions by 7 users not shown)
Line 3: Line 3:
The Global Meteor Network (GMN) is a world wide organization of amateur and professional astronomers alike, whose goal is to observe the night sky using low-light video cameras and produce meteor trajectories in a coordinated manner. Here you will find information on the purpose and structure of the GMN, assembling and operating your own meteor camera, contributing to the development of RMS the GMN software, as well as information on how your observations as a citizen scientist can contribute to the further understanding of our solar system's formation and evolution.  
The Global Meteor Network (GMN) is a world wide organization of amateur and professional astronomers alike, whose goal is to observe the night sky using low-light video cameras and produce meteor trajectories in a coordinated manner. Here you will find information on the purpose and structure of the GMN, assembling and operating your own meteor camera, contributing to the development of RMS the GMN software, as well as information on how your observations as a citizen scientist can contribute to the further understanding of our solar system's formation and evolution.  


'''<span style="color:red">If you have come here to find out how to build and setup a camera from scratch, jump ahead to [https://globalmeteornetwork.org/wiki/index.php?title=Main_Page#How_do_I_obtain_a_camera.3F this] section !</span>'''
'''<span style="color:red">If you have come here to find out how to build and setup a camera from scratch, jump ahead to [https://globalmeteornetwork.org/wiki/index.php?title=Build_%26_Install_%26_Setup_your_camera_-_The_complete_how-to this] section!</span>'''
 
'''<span style="color:red">For German speakers, there is the Build camera from scratch documentation written by students of [https://fsg-preetz.de/ Friedrich-Schiller-Gymnasium in Preetz] available [http://wiki.linux-astronomie.de/doku.php?id=ceres here]. This version is maintained by Friedrich-Schiller-Gymnasium in Preetz. </span>'''




Line 11: Line 13:


=== [https://globalmeteornetwork.org/?p=363 A brief history of the Global Meteor Network] ===
=== [https://globalmeteornetwork.org/?p=363 A brief history of the Global Meteor Network] ===
=== [https://www.youtube.com/watch?v=MAGq-XqD5Po Video introduction - Overview of the Global Meteor Network (IMC2020)] ===


=== [https://youtu.be/oM7lfQ4nmyw Video overview - Meteor tracking and the GMN from Astro Imaging Channel presentation] ===
=== [https://youtu.be/oM7lfQ4nmyw Video overview - Meteor tracking and the GMN from Astro Imaging Channel presentation] ===
Line 23: Line 27:
: Nightly, the RPi starts recording video from the camera shortly after local sunset continuously compressing and storing the video data locally. Each morning before sunrise, after capture is complete, the RPi analyzes the video and extracts your nightly station’s meteor observations. These extracted video “clips” of detected meteors are then archived and uploaded to a server. The clips can total hundreds of megabytes on a “busy” night (e.g., in a heavy meteor shower, or a night with a lot of false detections--progress is being made on the detection software). The server finds meteors which were observed with more than one station and this enables the server to triangulate the meteor trails in 3D and calculate the orbits of the meteors.
: Nightly, the RPi starts recording video from the camera shortly after local sunset continuously compressing and storing the video data locally. Each morning before sunrise, after capture is complete, the RPi analyzes the video and extracts your nightly station’s meteor observations. These extracted video “clips” of detected meteors are then archived and uploaded to a server. The clips can total hundreds of megabytes on a “busy” night (e.g., in a heavy meteor shower, or a night with a lot of false detections--progress is being made on the detection software). The server finds meteors which were observed with more than one station and this enables the server to triangulate the meteor trails in 3D and calculate the orbits of the meteors.


===[https://github.com/CroatianMeteorNetwork/RMS/blob/master/README.md  What do I need?] ===
=== What do I need? ===
 
You'll need a Raspberry Pi with the software on, and a camera kit.  We strongly recommend the Pi4 model. The software will run on a Pi3 but it is much slower and now also not maintained option. The Shopping list with everything you will need can be found on this [https://globalmeteornetwork.org/wiki/index.php?title=Shopping_list_and_tools_needed page].


You'll need a Raspberry Pi with the software on, and a camera kit. We strongly recommend the Pi4 model. The software will run on a Pi3 but it is much slower.  
It is also possible to run multiple cameras on a Linux PC. More details [https://docs.google.com/document/d/16PSFi8RAqbenPdluhulCRaIenOkEzgs5piUhkX3yaOc/edit here].


=== How do I obtain a camera? ===
=== How do I obtain a camera? ===
Line 31: Line 37:


==== Buy a Camera ====
==== Buy a Camera ====
You can buy a camera and Pi prebuilt and ready to install. These are available from a couple of suppliers. The Croatian Meteor Network sell prebuilt cameras as explained on [https://globalmeteornetwork.org/?page_id=136 this page]. Alternatively, if you're in the UK, you can obtain cameras from the [https://ukmeteornetwork.co.uk/ UK Meteor Network]  
You can buy a camera and Pi prebuilt and ready to install. These are available from a couple of suppliers. The Croatian Meteor Network sell prebuilt cameras as explained on [https://globalmeteornetwork.org/?page_id=136 this page]. Alternatively, if you're in the UK, you can obtain contact the [https://ukmeteornetwork.org/ UK Meteor Network] for advice, though as of 2024 they are unable to sell cameras directly.


==== Build your own from scratch ====
==== Build your own from scratch ====
This requires some basic DIY skills and some familiarity with the Raspberry PI, but don't be put off. The instructions are comprehensive and if you get stuck, you can ask for advice in the [https://groups.io/g/globalmeteornetwork groups.io] forum.
This requires some basic DIY skills and some familiarity with the Raspberry PI, but don't be put off. The instructions are comprehensive and if you get stuck, you can ask for advice in the [https://groups.io/g/globalmeteornetwork groups.io] forum.


Click on this link if you want to '''[[Build_A_Camera|build a camera from scratch]]'''.  
Click on this link if you want to '''[https://globalmeteornetwork.org/wiki/index.php?title=Build_%26_Install_%26_Setup_your_camera_-_The_complete_how-to build a camera from scratch]'''.


Once you've built the camera, You can  '''[https://globalmeteornetwork.org/wiki/index.php?title=Main_Page#Flash_a_pre-built_image flash a Prebuilt image]''' onto an SD card to set up the Pi with the software required to capture meteors and upload data to the network.
=== Advanced RMS installations and Multi-camera support ===
If you would like to explore some advanced possibilities to install RMS software onto various platforms or to run multiple cameras on a single Linux computer, please have a look at [https://globalmeteornetwork.org/wiki/index.php?title=Advanced_RMS_installations_and_Multi-camera_support this page]. In case you are planning to run RMS software on the Raspberry Pi 4, then please use the supported and easiest possibility by using the image we have prepared for you. More information is in [https://globalmeteornetwork.org/wiki/index.php?title=Build_%26_Install_%26_Setup_your_camera_-_The_complete_how-to this extensive guide].


=== Can I use a commercial all-sky camera? ===
=== Can I use a commercial all-sky camera? ===
Line 61: Line 68:
: Each morning you can review the result files on the RPi, and copy anything you want to your computer or tablet.
: Each morning you can review the result files on the RPi, and copy anything you want to your computer or tablet.


===Archiving data and backing up configuration ===
===Archiving data ===
 
: Your primary scientific data is automatically uploaded to the central server every morning when data processing is done. However once it has done this, RMS will purge out the oldest data to free up space for the next night's run.
 
: So, you may want to copy some of the data to a PC, NAS or cloud for further analysis of your own. The data you should consider backing up are the contents of '''~/RMS_data/ArchivedFiles''', which holds the individual files and data that RMS determined were probably meteors. Full detail on how to nbare beyond the scope of the GMN Wiki, but tools such as robocopy (for Windows) and rsync(for Linux/MacOS) are ideal. These tools can 'mirror' data across a network. If you want help configuring these, ask in the Globalmeteornetwork group on groups.io.
 
: We've also built some automated tools that can help to back up any additional data to a thumb drive inserted into the RPi. Please ask in the group about these.
 
===Backup and restore configuration and RSA keys===
 
:'''Please note:''' in case you are on older Buster image, then you need to replace username 'rms' with 'pi', e.g. /home/pi instead of /home/rms. How to determine which to use? Simply run:
::ls /home/rms home/pi
: whichever comes existent is your home directory
 
: Open a terminal and execute the command  '''Scripts/RMS_Backup.sh'''. A compressed .zip file containing all important configuration files and keys will be created in user's home directory with the prefix RMS_Backup and .zip extension. Example: ''/home/rms/RMS_Backup_XX0001_2023-01-28.zip''.
 
: Copy the .zip file to a safe place outside RPi, it will be useful later to restore the system in case of failure. Note it contains the RSA public and private keys used to contact GMN servers, keep it secret.
 
: To restore the configuration, unzip the backup file in some folder on the RPi and copy the files '''.config''', '''platepar_cmn2010.cal''' and '''mask.bmp''' to the folder '''/home/rms/source/RMS/''', and the files '''id_rsa''' and '''id_rsa.pub'' to the folder ''/home/rms/.ssh/''' as in the following example:
 
:: cp .config platepar_cmn2010.cal mask.bmp /home/rms/source/RMS/
:: cp id_rsa id_rsa.pub /home/rms/.ssh/
 
: Make sure that RSA key files permission bits are correct by using the command:


: Data backup is as much or as little as you like. Your primary data is automatically uploaded to the central server every morning when data processing is done. We've built some automated tools that can help to back up any additional data to a thumb drive inserted into the RPi.
:: chmod 400 ~/.ssh/id_rsa*


=== Viewing the data ===
=== Viewing the data ===


: To view the data, you can use CMN_binViewer software[https://github.com/CroatianMeteorNetwork/cmn_binviewer] which is already installed in the RMS SD image.  
: To view the data, you can use CMN_binViewer software[https://github.com/CroatianMeteorNetwork/cmn_binviewer] which is already installed in the RMS SD image.  
: There is also a Windows version[https://www.dropbox.com/s/44u5r9nso81wa56/CMN_binViewer_setup_x64.exe?dl=1] you can install.
: There is also a Windows version[https://github.com/CroatianMeteorNetwork/cmn_binviewer/releases] you can install.
 
: '''Important note''' : You can also open the images in astronomical FITS viewers such as FITS Liberator or Pixinsight, though the results may be surprising. For example in FITS Liberator, the image will be '''upside down'''. This is an artefact of how the software reads the image. In space, there's no 'up' or 'down' and so the FITS specification does not dictate whether the pixel (0,0) is at the bottom left or top left, or indeed one of the other corners. Some software, notably FITS Liberator, treats the top left as the origin and so terrestrial images will be displayed mirrored vertically.


=== Tools and Utilities ===
=== Tools and Utilities ===
Line 79: Line 111:
* You can run RMS Python jobs on the RPi to sample the image files captured all night long and condense them into a mp4 movie. This creates a sometimes mesmerizing summary that can run for over 2 minutes in length for winter time data.
* You can run RMS Python jobs on the RPi to sample the image files captured all night long and condense them into a mp4 movie. This creates a sometimes mesmerizing summary that can run for over 2 minutes in length for winter time data.


== Configuring and installing your camera ==
== What can I do with my GMN station? ==


=== [https://docs.google.com/document/u/1/d/1MTL85YMYH33u8OZ9Tl0JjSZXcLn8mOmNrCOC0I28Drw/edit Quick start guide] ===
=== Using SkyFit2 to perform astrometric and photometric calibration + Manually reducing observations of fireballs and computing their trajectories: ===
* [https://www.youtube.com/watch?v=ao3J9Jf0iLQ Updated 2023 video tutorial]
* [https://www.youtube.com/watch?v=MOjb3qxDlX4 Old 2021 video tutorial]


=== [[Focusing_your_camera | Focusing your IP camera]] ===
=== [https://globalmeteornetwork.org/fov3d/ Generating a Google Earth KML file to show your station's field of view] ===


=== [https://www.youtube.com/watch?v=N2sq1hBwcAA Video tutorial - Configuring your IMX291 IP camera] ===
=== [https://globalmeteornetwork.org/?p=253 Using UFO Orbit program to estimate meteor trajectories] ===


=== [https://docs.google.com/document/d/1TPotXcRstHz-XWIQYIW71xkRYy7Ca1UCoEiZsqdk9zw/edit?usp=sharing Different ways of connecting the camera] ===
=== [https://globalmeteornetwork.org/?p=221 Urban meteor observing] ===


== What can I do with my GMN station? ==
== Data analysis with SkyFit2 ==


=== [https://www.youtube.com/watch?v=MOjb3qxDlX4 Video Tutorial - Using SkyFit2 to perform astrometric and photometric calibration + Manually reducing observations of fireballs and computing their trajectories] ===
SkyFit2 is a program within the RMS library which supports analyzing optical meteor data in most optical formats that are in use today, including videos in any popular video format (mp4, avi, mkv), a sequence of static images, or a single image with shutter breaks.


=== [https://globalmeteornetwork.org/?p=310 Generating a Google Earth KML file to show your station's field of view] ===
This [https://www.youtube.com/watch?v=ao3J9Jf0iLQ video tutorial] explains how to using SkyFit2 to perform astrometric and photometric calibration on GMN data and manually reduce observations of fireballs, including computing their trajectories.


=== [https://globalmeteornetwork.org/?p=253 Using UFO Orbit program to estimate meteor trajectories] ===
A more general and detailed description of SkyFit2 is given at the '''[[SkyFit2|SkyFit2]]''' page.


=== [https://globalmeteornetwork.org/?p=221 Urban meteor observing] ===
== FAQ ==


== RMS Software Installation ==
=== What should I back up when re-flashing an SD card or a USB disk? ===


=== Flash a pre-built image ===
: The .config, platepar and mask files that are in the RMS source directory, plus the whole contents of the hidden directory /home/pi/.ssh.


Pre-built Images are available with the RMS software pre-installed on Raspbian. Raspbian is the operating system typically used on the Raspberry Pi.  
If your SD card or a USB disk fails or becomes corrupted, the config files can be fetched from the server as they are uploaded every day together with the data. However the contents of .ssh are essential for connection to the server, so you must also save these. Once you set up a new SD card or a USB disk, return the files in their original location.


A 128 GB microSD card is preferred, although smaller 64 GB cards will also work. The image is flashed to the microSD card using the Etcher utility for Windows (32 or 64-bit), MacOS or Linux (64-bit), which can be found [https://www.balena.io/etcher/ here]
=== What are the values in FTPdetectinfo_*  file designated as hnr mle bin Pix/fm Rho Phi? ===


When you first boot up the Pi make sure you have a keyboard, monitor and mouse attached as you will be taken through a setup process by the RMS software.  
:  Some of these values are not used in RMS (hnr mle bin), but they are in CAMS, so they are here to conform to the standard. Thus they are all zeros. The others are:


The current publicly released images of RMS are:
: - Pix/fm - Average angular speed of the meteor in pixels per frame.
[https://www.dropbox.com/s/sa9csd4m3jr1vqo/RMS_image_20191203.zip?dl=1 RPi3B+ ]
and
[https://www.dropbox.com/s/a6ix7nz9f27h03d/RMS_RPi4_image_20200604.zip?dl=1 RPi4 ].  


Please see the [https://docs.google.com/document/u/1/d/1MTL85YMYH33u8OZ9Tl0JjSZXcLn8mOmNrCOC0I28Drw/edit Quick Start Guide] about how to flash them to an SD card.
: - Rho, Phi - Parameters that define the line of the meteor in polar coordinates, see [https://en.wikipedia.org/wiki/Hough_transform#Theory here] for more details. Rho is the distance of the line from the centre of the image, and phi is the angle of the line as measured from the positive direction of the Y axis (basically a line going from the center of the image to the top of the image), the positive angles are measured clockwise (I think, the CAMS standard might define these parameters a bit differently, the Y axis is flipped).


Notes:  
: The intensity is the sum of all pixel intensities of the meteor on a given frame. Let's say I represent an area around the meteor on a given frame like this, where the numbers are pixel intensities on an 8-bit image (so they can range from 0 to 255):
* The earlier versions of the Raspberry Pi, the 1 and 2, are not powerful enough to run RMS software.


* the pre-installed RMS software images incorporate an auto-updating feature, so that the RMS software is updated to the current release whenever your Raspberry Pi RMS is booted. This way, your station is always running the most recent set of updates.  
: [[File:Intensity_sum.png |Intensity_sum.png ]]


* For installations into other Linux or windows environments, executing the command ''git pull'' will update to the most recent RMS release.
: and the pixels values inside the red boundary represent the meteor blob on the frame, the intensity would be the sum of all numbers inside the red boundary.
: This value is later used to compute the magnitude. The magnitude is computed as: mag = -2.5*log10(intensity sum) + photometric_offset. The photometric offset is estimated in SkyFit by fitting the line with slope -2.5 through pairs of known magnitudes of stars and logartihms of their pixel intensity sum. The photometric offset is basically the intercept of that line. The constant slope of -2.5 comes from the [https://en.wikipedia.org/wiki/Apparent_magnitude#Calculations definition of stellar magnitudes.]


* The code was designed to run on a RPi, but it will also run an some Linux distributions. We have tested it on Linux Mint 18 and Ubuntu 16. For information on installing into other Linux releases, check portions of the section listed below for installing on RPi, and also check installation instructions found on [https://github.com/CroatianMeteorNetwork/RMS GitHub]


=== Build the software from scratch (not for the faint-hearted) ===
== GMN data policy ==


If you'd like to install RMS on the RPi from scratch (not recommended), you can follow these installation instructions:
The Global Meteor Network produces several levles of data products:
* Level 1 - The lowest level data (i.e. as close to "raw" as possible) are the FF image and FR video files saved to the RPi by the capture code and the fireball detector.
* Level 2 - The meteor detector uses these data to extract positional and brightness information of individual meteors (FTPdetectinfo file), and images are also used for astrometric and photometric calibration (platepar file).  Meteor and star detections are used to generate a range of plots suchs as the single-station shower association graph, camera drift graph, etc. The calibrated meteors measurements get uploaded to the GMN server together with the raw images of individual meteors.
* Level 3 - The software on the server correlates individual observations and computes multi-station meteor trajectories which are published daily on the GMN [https://globalmeteornetwork.org/data/ data website]. This data is made public under the [https://creativecommons.org/licenses/by/4.0/ CC BY 4.0 license].


=== [https://docs.google.com/document/d/1ChaXx6WkuGfcQ_f5yRbwUOOMxEgIJrjq932UZE6cRjE/edit Install for Raspberry Pi 3] ===
Operators of individual GMN stations exclusively own the Level 1 and Level 2 data their stations produce. In practice, this means that they are free to share this data with other meteor networks if they wish to do so. The data that gets uploaded to the GMN server will not be shared publicly nor with other parties without the operator's consent, but may be used internally by the GMN coordinators to manually produce other data products (e.g. trajectory of a meteorite dropping fireball, analysis of a meteor shower). All station operators will be credited for their data in all GMN publications.
=== [https://docs.google.com/document/d/19ImeNqBTD1ml2iisp5y7CjDrRV33wBeF9rtx3mIVjh4/edit?usp=sharing Install for Raspberry Pi 4] ===


=== Installation for Windows ===
== IstraStream ==
Note that installation for Windows is optional. However, if you want to run RMS tools on your PC, where its much quicker to run SkyFit2 to calibrate your camera to the sky, or you want to use BatchFFToImage to convert data to jpeg format to share on Social Media, Click '''[[Windows_Installation|here]]''' for instructions.


Note that data capture and meteor detection does not work under windows! If you are a Windows whizz, please feel free to investigate why this doesn't work.  
=== [http://istrastream.com/rms-gmn/ IstraStream GMN status website NOW OBSOLETE] ===


Finally: these instructions have been tested on Windows 10 64-bit but should work for any version of Windows that Anaconda and Git are available for.
: IstraStream.com is an independent hosting site primarily for cameras sold by IstraStream. In mid-2023 Istrastream stopped listing camera image output and the IstraStream data display has been replaced by the [https://globalmeteornetwork.org/weblog/ GMN weblog].


=== Installation for MacOS ===
This document explains what every plot on the IstraStream weblog means:
Installation for MacOS is optional as it is for Windows. If you do want to install it, please follow the instructions '''[[MacOS_Install | here]]'''.


== FAQ ==
=== [https://docs.google.com/document/d/132aHGn0QPzhpVN2s2n6FT6rJn39LAsPkchWJqXQb8Qk/edit?usp=sharing IstraStream Plots Explanation] ===
 
== For More Information ==


=== What should I back up when re-flashing an SD card? ===
=== [https://globalmeteornetwork.org/?page_id=43 Contact the Global Meteor Network] ===


: The .config, platepar and mask files that are in the RMS source directory, plus the whole contents of the hidden directory /home/pi/.ssh.
=== [https://groups.io/g/globalmeteornetwork Join the Global Meteor Network Forum] ===


If your SD card fails or becomes corrupted, the config files can be fetched from the server as they are uploaded every day together with the data. However the contents of .ssh are essential for connection to the server, so you must also save these
=== [https://github.com/markmac99/ukmon-pitools/wiki UK Meteor Network Wiki]===
This wiki has numerous FAQs and tips on maintaining, monitoring and managing your system, and several explainers such as how to calibrate and create a mask, how to copy data and so forth.


=== What are the values in FTPdetectinfo_*  file designated as hnr mle bin Pix/fm Rho Phi? ===
=== GMN talks ===


: Some of these values are not used in RMS (hnr mle bin), but they are in CAMS, so they are here to conform to the standard. Thus they are all zeros. The others are:
: [https://www.youtube.com/playlist?list=PLmQ5Bvz4ACYJLYfswIeAipapoeGeI6QWy GMN talk for Society for Astronomical Sciences workshop 2024 (The first 3 videos)]


: - Pix/fm - Average angular speed of the meteor in pixels per frame.
: [https://www.youtube.com/watch?v=IfUyCHjMATc 2023 GMN Meeting Session 1 (February 2023)]


: - Rho, Phi - Parameters that define the line of the meteor in polar coordinates, see [https://en.wikipedia.org/wiki/Hough_transform#Theory here] for more details. Rho is the distance of the line from the centre of the image, and phi is the angle of the line as measured from the positive direction of the Y axis (basically a line going from the center of the image to the top of the image), the positive angles are measured clockwise (I think, the CAMS standard might define these parameters a bit differently, the Y axis is flipped).
: [https://www.youtube.com/watch?v=I78KwF5-1GE 2023 GMN Meeting Session 2 (February 2023)]


: The intensity is the sum of all pixel intensities of the meteor on a given frame. Let's say I represent an area around the meteor on a given frame like this, where the numbers are pixel intensities on an 8-bit image (so they can range from 0 to 255):
: [https://www.youtube.com/watch?v=wDdrG_FCyGk 2022 GMN Meeting Session 1 (February 2022)]


: [[File:Intensity_sum.png |Intensity_sum.png ]]
: [https://www.youtube.com/watch?v=j_75CDPzjI4 2022 GMN Meeting Session 2 (February 2022)]


: and the pixels values inside the red boundary represent the meteor blob on the frame, the intensity would be the sum of all numbers inside the red boundary.
: [https://www.youtube.com/watch?v=f6x9_WCVphY GMN talk at the European Space Agency's Fireball Workshop (June 2021)]
: This value is later used to compute the magnitude. The magnitude is computed as: mag = -2.5*log10(intensity sum) + photometric_offset. The photometric offset is estimated in SkyFit by fitting the line with slope -2.5 through pairs of known magnitudes of stars and logartihms of their pixel intensity sum. The photometric offset is basically the intercept of that line. The constant slope of -2.5 comes from the [https://en.wikipedia.org/wiki/Apparent_magnitude#Calculations definition of stellar magnitudes.]


== IstraStream ==
: [https://www.youtube.com/watch?v=QXBTLPnPDWs 2021 GMN Meeting] - [https://www.dropbox.com/sh/ia9vagug5lxm8k9/AAB_i_1jcWThUdAHO_2gF_Ksa?dl=0 Link to slides]
: The IstraStream.com is an independent hosting site which is part of our world wide GMN. It is primarily a host for data from cameras sold by IstraStream, but other station operators can request that their data be included. At this time, data is hosted from RMS stations in over 10 countries, with a geographic span including Europe, North and South America.
If you want to upload your data to IstraStream, contact [mailto:info@istrastream.com info@istrastream.com]


=== [http://istrastream.com/rms-gmn/ IstraStream GMN status website] ===
: [https://www.youtube.com/watch?v=MAGq-XqD5Po Overview of the GMN - IMC2020 (September 2020)]


=== [https://docs.google.com/document/d/132aHGn0QPzhpVN2s2n6FT6rJn39LAsPkchWJqXQb8Qk/edit?usp=sharing IstraStream Plots Explanation] ===
: [https://www.youtube.com/watch?v=oM7lfQ4nmyw Overview of the GMN, Astro Imaging Channel presentation (May 2020)]


== For More Information ==
=== GMN-related publications ===


=== [https://globalmeteornetwork.org/?page_id=43 Contact the Global Meteor Network] ===
: [https://arxiv.org/abs/2206.11365 Vida, D., Blaauw Erskine, R. C., Brown, P. G., Kambulow, J., Campbell-Brown, M., & Mazur, M. J. (2022). Computing optical meteor flux using global meteor network data. Monthly Notices of the Royal Astronomical Society, 515(2), 2322-2339.]


=== [https://groups.io/g/globalmeteornetwork Join the Global Meteor Network Forum] ===
: [https://academic.oup.com/mnras/advance-article/doi/10.1093/mnras/stab2557/6368869 Moorhead, A. V., Clements, T., & Vida, D. (2021). Meteor shower radiant dispersions in Global Meteor Network data. Monthly Notices of the Royal Astronomical Society, 508(1), 326-339.]


=== GMN related publications ===
: [https://arxiv.org/abs/2107.12335 Vida, D., Šegon, D., Gural, P. S., Brown, P. G., McIntyre, M. J., Dijkema, T. J., Pavletić, L., Kukić, P., Mazur, M.J., Eschman, P., Roggemans, P., Merlak, A., & Zubović, D. (2021). The Global Meteor Network–Methodology and first results. Monthly Notices of the Royal Astronomical Society, 506(4), 5046-5074.]


: [https://arxiv.org/abs/2003.05458/ Moorhead, A. V., Clements, T. D., & Vida, D. (2020). Realistic gravitational focusing of meteoroid streams. Monthly Notices of the Royal Astronomical Society, 494(2), 2982-2994.]  
: [https://arxiv.org/abs/2003.05458/ Moorhead, A. V., Clements, T. D., & Vida, D. (2020). Realistic gravitational focusing of meteoroid streams. Monthly Notices of the Royal Astronomical Society, 494(2), 2982-2994.]  

Latest revision as of 16:20, 28 September 2024

Welcome to the Global Meteor Network's wiki page!

The Global Meteor Network (GMN) is a world wide organization of amateur and professional astronomers alike, whose goal is to observe the night sky using low-light video cameras and produce meteor trajectories in a coordinated manner. Here you will find information on the purpose and structure of the GMN, assembling and operating your own meteor camera, contributing to the development of RMS the GMN software, as well as information on how your observations as a citizen scientist can contribute to the further understanding of our solar system's formation and evolution.

If you have come here to find out how to build and setup a camera from scratch, jump ahead to this section!

For German speakers, there is the Build camera from scratch documentation written by students of Friedrich-Schiller-Gymnasium in Preetz available here. This version is maintained by Friedrich-Schiller-Gymnasium in Preetz.


Global Meteor Network Overview

Our mission

A brief history of the Global Meteor Network

Video introduction - Overview of the Global Meteor Network (IMC2020)

Video overview - Meteor tracking and the GMN from Astro Imaging Channel presentation

Some 'live' GMN data products

Meteor Detection Station

What is an RMS GMN station?

A RMS-based GMN station that is the subject of this Wiki consists of a Raspberry Pi (RPi) single board computer, a low light level security video camera, and the RMS software package. The camera is securely mounted in a weatherproof housing, pointed at the sky, and connected to the RPi with a POE (Power Over Ethernet) cable. The RPi is connected to the Internet via WiFi, and to be a part of GMN network, you’ll need a fairly powerful Raspberry Pi (RPi 3B+, RPi 4 or better) and a reasonably fast Internet connection. The internet connection is primarily required to enable data upload to a central server each morning as well as provide automatic updates for the RMS software.
Nightly, the RPi starts recording video from the camera shortly after local sunset continuously compressing and storing the video data locally. Each morning before sunrise, after capture is complete, the RPi analyzes the video and extracts your nightly station’s meteor observations. These extracted video “clips” of detected meteors are then archived and uploaded to a server. The clips can total hundreds of megabytes on a “busy” night (e.g., in a heavy meteor shower, or a night with a lot of false detections--progress is being made on the detection software). The server finds meteors which were observed with more than one station and this enables the server to triangulate the meteor trails in 3D and calculate the orbits of the meteors.

What do I need?

You'll need a Raspberry Pi with the software on, and a camera kit. We strongly recommend the Pi4 model. The software will run on a Pi3 but it is much slower and now also not maintained option. The Shopping list with everything you will need can be found on this page.

It is also possible to run multiple cameras on a Linux PC. More details here.

How do I obtain a camera?

There are two options:

Buy a Camera

You can buy a camera and Pi prebuilt and ready to install. These are available from a couple of suppliers. The Croatian Meteor Network sell prebuilt cameras as explained on this page. Alternatively, if you're in the UK, you can obtain contact the UK Meteor Network for advice, though as of 2024 they are unable to sell cameras directly.

Build your own from scratch

This requires some basic DIY skills and some familiarity with the Raspberry PI, but don't be put off. The instructions are comprehensive and if you get stuck, you can ask for advice in the groups.io forum.

Click on this link if you want to build a camera from scratch.

Advanced RMS installations and Multi-camera support

If you would like to explore some advanced possibilities to install RMS software onto various platforms or to run multiple cameras on a single Linux computer, please have a look at this page. In case you are planning to run RMS software on the Raspberry Pi 4, then please use the supported and easiest possibility by using the image we have prepared for you. More information is in this extensive guide.

Can I use a commercial all-sky camera?

Generally no due to the lack of sensitivity. But see this recent experiment

Operating and maintaining your GMN station

Overview

Please note that GMS is a nascent operation and you may share some growing pains if you choose to be involved -- we're still working out some bugs and making improvements here, which may be an opportunity to help if you have programming skills! ;-) So note that the workload of day-to-day operation can be non-zero, and might take a little bit of your time.
Ideally, you'll want to monitor your RMS RPi system(s) daily to look for freezes or glitches or other problems... like birds nesting or soiling the camera window, people accidentally unplugging the power cord, mice (or cats or dogs!) chewing on the camera Ethernet cable, etc.
Although we are getting close, this is not a "power up and forget about it" system yet.
However, by its very nature, the GMS network is inhabited by a lot of people who are willing to help newcomers getting started. So, here are some clues for daily operation of your RMS camera.

So what does the meteor camera do over the course of 24 hours?

The RMS Python based system calculates the sundown to sunrise interval and schedules video camera capture all night long. Depending on the video camera and capabilities of the RPi, the camera captures 25 or more frames per second between evening and morning twilight. During the continuous image capture, the station begins processing captured image data, doing a pre-screening to target frames with a suitable number of stars (usually around 20) that makes it worth looking for meteor detections. Once data capture has finished, the station switches into processing all the promising frames for detections, then refining the astrometric accuracy of every positive detection. Using the station platepar (plate parameters) calibration file, processing iterates to find the best astrometry and photometry solution for each detected meteor. Once this process has analyzed each detection, summary files are created.
These summary files include text file data presentation in several widely accepted formats (CAMS and UFOorbit), as well as graphic plots of detection frequencies throughout the night, a set of thumbnail images of detections, a set of thumbnail images of data captured throughout the night, a single image with all detections stacked together, plots of photometry, astrometry, and camera pointing drift in arc minutes throughout the course of the night as the mount or building flexes, a flat file for correcting images, and a plot of all detections showing any identified radiants. Finally all results are combined into a single compressed archive, which is automatically uploaded each morning to the central server. Optionally, you can create a mp4 movie showing a time lapse of the night’s captured images.
Each morning you can review the result files on the RPi, and copy anything you want to your computer or tablet.

Archiving data

Your primary scientific data is automatically uploaded to the central server every morning when data processing is done. However once it has done this, RMS will purge out the oldest data to free up space for the next night's run.
So, you may want to copy some of the data to a PC, NAS or cloud for further analysis of your own. The data you should consider backing up are the contents of ~/RMS_data/ArchivedFiles, which holds the individual files and data that RMS determined were probably meteors. Full detail on how to nbare beyond the scope of the GMN Wiki, but tools such as robocopy (for Windows) and rsync(for Linux/MacOS) are ideal. These tools can 'mirror' data across a network. If you want help configuring these, ask in the Globalmeteornetwork group on groups.io.
We've also built some automated tools that can help to back up any additional data to a thumb drive inserted into the RPi. Please ask in the group about these.

Backup and restore configuration and RSA keys

Please note: in case you are on older Buster image, then you need to replace username 'rms' with 'pi', e.g. /home/pi instead of /home/rms. How to determine which to use? Simply run:
ls /home/rms home/pi
whichever comes existent is your home directory
Open a terminal and execute the command Scripts/RMS_Backup.sh. A compressed .zip file containing all important configuration files and keys will be created in user's home directory with the prefix RMS_Backup and .zip extension. Example: /home/rms/RMS_Backup_XX0001_2023-01-28.zip.
Copy the .zip file to a safe place outside RPi, it will be useful later to restore the system in case of failure. Note it contains the RSA public and private keys used to contact GMN servers, keep it secret.
To restore the configuration, unzip the backup file in some folder on the RPi and copy the files .config, platepar_cmn2010.cal and mask.bmp to the folder /home/rms/source/RMS/, and the files id_rsa and id_rsa.pub to the folder /home/rms/.ssh/ as in the following example:
cp .config platepar_cmn2010.cal mask.bmp /home/rms/source/RMS/
cp id_rsa id_rsa.pub /home/rms/.ssh/
Make sure that RSA key files permission bits are correct by using the command:
chmod 400 ~/.ssh/id_rsa*

Viewing the data

To view the data, you can use CMN_binViewer software[1] which is already installed in the RMS SD image.
There is also a Windows version[2] you can install.
Important note : You can also open the images in astronomical FITS viewers such as FITS Liberator or Pixinsight, though the results may be surprising. For example in FITS Liberator, the image will be upside down. This is an artefact of how the software reads the image. In space, there's no 'up' or 'down' and so the FITS specification does not dictate whether the pixel (0,0) is at the bottom left or top left, or indeed one of the other corners. Some software, notably FITS Liberator, treats the top left as the origin and so terrestrial images will be displayed mirrored vertically.

Tools and Utilities

  • RealVNC or AnyDesk remote connect tool allows station access from anywhere. Access from outside your network is enabled by use of an OpenVPN connection address available to meteor stations. Alternatively, with VNC and Teamviewer, you can create an account and team on their websites, and then remotely access your station.
  • Samba data directory access, allows you to copy data results directly from your RPi to your computer or tablet.
  • CMN_binViewer can be used to view standard fits image files containing meteor detections. It runs on the RPi, and is also available under Windows.
  • UFO Orbit allows you to process data from multiple stations and generate unified radiants of two or more stations seeing the same meteor. It can plot the shared object ground path, orbital characteristics, and can output a summary file of all objects seen by more than one station, which can be used for further analysis.
  • RMS software can be installed under Windows to allow much of the RMS python-based code to be executed on your computer, so it can be run against meteor station data you have transferred to your computer from the RPi.
  • You can run RMS Python jobs on the RPi to sample the image files captured all night long and condense them into a mp4 movie. This creates a sometimes mesmerizing summary that can run for over 2 minutes in length for winter time data.

What can I do with my GMN station?

Using SkyFit2 to perform astrometric and photometric calibration + Manually reducing observations of fireballs and computing their trajectories:

Generating a Google Earth KML file to show your station's field of view

Using UFO Orbit program to estimate meteor trajectories

Urban meteor observing

Data analysis with SkyFit2

SkyFit2 is a program within the RMS library which supports analyzing optical meteor data in most optical formats that are in use today, including videos in any popular video format (mp4, avi, mkv), a sequence of static images, or a single image with shutter breaks.

This video tutorial explains how to using SkyFit2 to perform astrometric and photometric calibration on GMN data and manually reduce observations of fireballs, including computing their trajectories.

A more general and detailed description of SkyFit2 is given at the SkyFit2 page.

FAQ

What should I back up when re-flashing an SD card or a USB disk?

The .config, platepar and mask files that are in the RMS source directory, plus the whole contents of the hidden directory /home/pi/.ssh.

If your SD card or a USB disk fails or becomes corrupted, the config files can be fetched from the server as they are uploaded every day together with the data. However the contents of .ssh are essential for connection to the server, so you must also save these. Once you set up a new SD card or a USB disk, return the files in their original location.

What are the values in FTPdetectinfo_* file designated as hnr mle bin Pix/fm Rho Phi?

Some of these values are not used in RMS (hnr mle bin), but they are in CAMS, so they are here to conform to the standard. Thus they are all zeros. The others are:
- Pix/fm - Average angular speed of the meteor in pixels per frame.
- Rho, Phi - Parameters that define the line of the meteor in polar coordinates, see here for more details. Rho is the distance of the line from the centre of the image, and phi is the angle of the line as measured from the positive direction of the Y axis (basically a line going from the center of the image to the top of the image), the positive angles are measured clockwise (I think, the CAMS standard might define these parameters a bit differently, the Y axis is flipped).
The intensity is the sum of all pixel intensities of the meteor on a given frame. Let's say I represent an area around the meteor on a given frame like this, where the numbers are pixel intensities on an 8-bit image (so they can range from 0 to 255):
Intensity_sum.png
and the pixels values inside the red boundary represent the meteor blob on the frame, the intensity would be the sum of all numbers inside the red boundary.
This value is later used to compute the magnitude. The magnitude is computed as: mag = -2.5*log10(intensity sum) + photometric_offset. The photometric offset is estimated in SkyFit by fitting the line with slope -2.5 through pairs of known magnitudes of stars and logartihms of their pixel intensity sum. The photometric offset is basically the intercept of that line. The constant slope of -2.5 comes from the definition of stellar magnitudes.


GMN data policy

The Global Meteor Network produces several levles of data products:

  • Level 1 - The lowest level data (i.e. as close to "raw" as possible) are the FF image and FR video files saved to the RPi by the capture code and the fireball detector.
  • Level 2 - The meteor detector uses these data to extract positional and brightness information of individual meteors (FTPdetectinfo file), and images are also used for astrometric and photometric calibration (platepar file). Meteor and star detections are used to generate a range of plots suchs as the single-station shower association graph, camera drift graph, etc. The calibrated meteors measurements get uploaded to the GMN server together with the raw images of individual meteors.
  • Level 3 - The software on the server correlates individual observations and computes multi-station meteor trajectories which are published daily on the GMN data website. This data is made public under the CC BY 4.0 license.

Operators of individual GMN stations exclusively own the Level 1 and Level 2 data their stations produce. In practice, this means that they are free to share this data with other meteor networks if they wish to do so. The data that gets uploaded to the GMN server will not be shared publicly nor with other parties without the operator's consent, but may be used internally by the GMN coordinators to manually produce other data products (e.g. trajectory of a meteorite dropping fireball, analysis of a meteor shower). All station operators will be credited for their data in all GMN publications.

IstraStream

IstraStream GMN status website NOW OBSOLETE

IstraStream.com is an independent hosting site primarily for cameras sold by IstraStream. In mid-2023 Istrastream stopped listing camera image output and the IstraStream data display has been replaced by the GMN weblog.

This document explains what every plot on the IstraStream weblog means:

IstraStream Plots Explanation

For More Information

Contact the Global Meteor Network

Join the Global Meteor Network Forum

UK Meteor Network Wiki

This wiki has numerous FAQs and tips on maintaining, monitoring and managing your system, and several explainers such as how to calibrate and create a mask, how to copy data and so forth.

GMN talks

GMN talk for Society for Astronomical Sciences workshop 2024 (The first 3 videos)
2023 GMN Meeting Session 1 (February 2023)
2023 GMN Meeting Session 2 (February 2023)
2022 GMN Meeting Session 1 (February 2022)
2022 GMN Meeting Session 2 (February 2022)
GMN talk at the European Space Agency's Fireball Workshop (June 2021)
2021 GMN Meeting - Link to slides
Overview of the GMN - IMC2020 (September 2020)
Overview of the GMN, Astro Imaging Channel presentation (May 2020)

GMN-related publications

Vida, D., Blaauw Erskine, R. C., Brown, P. G., Kambulow, J., Campbell-Brown, M., & Mazur, M. J. (2022). Computing optical meteor flux using global meteor network data. Monthly Notices of the Royal Astronomical Society, 515(2), 2322-2339.
Moorhead, A. V., Clements, T., & Vida, D. (2021). Meteor shower radiant dispersions in Global Meteor Network data. Monthly Notices of the Royal Astronomical Society, 508(1), 326-339.
Vida, D., Šegon, D., Gural, P. S., Brown, P. G., McIntyre, M. J., Dijkema, T. J., Pavletić, L., Kukić, P., Mazur, M.J., Eschman, P., Roggemans, P., Merlak, A., & Zubović, D. (2021). The Global Meteor Network–Methodology and first results. Monthly Notices of the Royal Astronomical Society, 506(4), 5046-5074.
Moorhead, A. V., Clements, T. D., & Vida, D. (2020). Realistic gravitational focusing of meteoroid streams. Monthly Notices of the Royal Astronomical Society, 494(2), 2982-2994.
Kukić, P., Gural, P., Vida, D., Šegon, D. & Merlak, A. (2018) Correction for meteor centroids observed using rolling shutter cameras. WGN, Journal of the International Meteor Organization, 46:5, 154-118.
Vida, D., Mazur, M. J., Šegon, D., Kukić, P., & Merlak, A. (2018). Compressive strength of a skirting Daytime Arietid-first science results from low-cost Raspberry Pi-based meteor stations. WGN, Journal of the International Meteor Organization, 46, 113-118.
Vida, D., Gural, P., Brown, P., Campbell-Brown, M., Wiegert, P. (2019) Estimating trajectories of meteors: an observational Monte Carlo approach - I. Theory. arXiv:1911.02979v4 [astro-ph.EP 21 Apr 2020]
Vida, D., Gural, P., Brown, P., Campbell-Brown, M., Wiegert, P. (2019) Estimating trajectories of meteors: an observational Monte Carlo approach - II. Results. arXiv:1911.11734v1 [astro-ph.EP 26 Novr 2019]
Vida, D., Mazur, M. J., Šegon, D., Zubović, D., Kukić, P., Parag, F., & Macan, A. (2018). First results of a Raspberry Pi based meteor camera system. WGN, Journal of the International Meteor Organization, 46, 71-78.
Vida, D., Zubović, D., Šegon, D., Gural, P., & Cupec, R. (2016). Open-source meteor detection software for low-cost single-board computers. In Proceedings of the International Meteor Conference (IMC2016), Egmond, The Netherlands (pp. 2-5).
Zubović, D., Vida, D., Gural, P., & Šegon, D. (2015). Advances in the development of a low-cost video meteor station. In Proceedings of the International Meteor Conference, Mistelbach, Austria (pp. 27-30).