User:AlliterativeAnchovies/Rover Progress Image Generation

Want to make or edit this file?

That's totally fine; I’m lazy so this page isn’t always up to date with the code used to generate the file, so leave a message on my talk page and I’ll update this for you :)

I can only update the graph after NASA updates their “where is Perseverance” page; this can take a couple days or even a week or two after Ingenuity flies, so I can’t update it immediately after flight. However I do check the “where is Perseverance” page at least once every 2 days after a flight occurs, so once the page is updated, the image will be updated promptly.

Current status

Current status of the graph.

Current status of curiosity graph

How to get raw data

Automatically Getting the Data

NASA hosts the relevant json files at certain urls, although it can be difficult to find those urls. The json data that defines the “Where is Perseverance/Curiosity” page is available here (Percy) and here (Curi). Within those pages you can find all the links to the specific layers of the “Where is ____” pages, with the relevant specific datasets being the Curiosity Waypoints, Perseverance Waypoints, and Ingenuity Flight Path. Only the last three files are strictly necessary, the first two ones I linked are just meant to be used as a directory in case you want to find more related datasets. All of these urls can just be pulled directly by an automated program and read as json files (which my code then turns into Pandas tables).

Unfortunately, we can use a related link to see that there is no direct analog of this data for the other NASA rovers (Sojourner, Spirit, Opportunity), as only MSL (“Mars Science Laboratory”, i.e. Curiosity) and M20 (“Mars 2020”, i.e. Perseverance) data is available. If you know of a place to get this data, please let me know (and also for Zhurong; I can't read Chinese so it is hard for me to investigate)! I'm looking into using NASA's SPICE data but it seems complicated so it may take a while, especially as I'm fairly busy irl these days.

Manually Getting the Data

Go to the Perseverance location tracker page[1], click on the top right button that looks like three sheets stacked on top of eachother. Click on ‘Waypoints’, then click on the download button, which will make the ‘Export GeoJSON’ button pop up - click it to get your data. Name it PerseveranceWaypoints.geojson.json. This works with Curiosity and ingenuity too. If you want to include them, make sure to set the right variables in the code.

For Ingenuity, you want the “Helicopter Flight Path” data, not “Helicopter Waypoints”.

Side note

For Ingenuity, the data is stored on a daily instead of total basis. However, the precision seems to be hundredths of a meter, so there shouldn’t be a discernible drift error from just adding these numbers up; this accuracy is also probably on the edge of what is feasibly possible to get on Mars even for NASA, centimeter precision is really good on Earth too!

Code

Run this code to get the image to generate. I originally wrote it in a Jupiter Notebook, so it might need some tweaking to work as a standalone. Note that the version here is pretty out of date, and requires manual download of datasets. The current version, on my local machine, is better for a few reasons, so you might want to leave me a message on my talk page asking me to update this if you want to use it.

#!/usr/bin/env python
# coding: utf-8

# In[184]:


import pandas as pd
import numpy as np
import json
import matplotlib.pyplot as plt
get_ipython().run_line_magic('matplotlib', 'notebook')


# In[185]:

rover_name = 'Curiosity'
waypoints_filename = f'./{rover_name}Waypoints.geojson.json'
helicopter_filename = 'Helicopter Flight Path.geojson.json'

use_helicopter = True

with open(waypoints_filename) as json_file:
    waypoints_json = json.load(json_file)
    
with open(helicopter_filename) as json_file:
    helicopter_json = json.load(json_file)

waypoints_df = pd.json_normalize(waypoints_json['features'])
helicopter_df = pd.json_normalize(helicopter_json['features'])


# In[186]:


# Curiosity doesn't have the dist_total field, instead it is
# called dist_km, but represents the same thing
if 'properties.dist_total' not in waypoints_df.columns:
    waypoints_df['properties.dist_total'] = (
        waypoints_df['properties.dist_km'].astype('float64')
    )

# Make right datatype
waypoints_df['properties.sol'] = waypoints_df['properties.sol'].astype('int64')
    
waypoints_df.head()


# In[187]:


# There is a wrong value in Curiosity's dataset,
# where the total distance is listed as 0 a couple
# hundred sols into the mission for one data point.
# So to fix this, wherever the data is 0 after the first
# nonzero value, we replace it with the previous value.

# Replaze zeros with nan
waypoints_df.loc[waypoints_df['properties.dist_total'] <= 0] = np.nan

# Put back first zero
waypoints_df.loc[0, 'properties.dist_total'] = 0

# Replace nans with previous value
waypoints_df.ffill(inplace=True)


# In[188]:


# Ensure right datatypes
helicopter_df['properties.sol'] = helicopter_df['properties.sol'].astype('int64')
helicopter_df['properties.length'] = helicopter_df['properties.length'].astype('float64')

helicopter_df


# In[189]:


# Get just the relevant info
dist_sol_df = waypoints_df[['properties.dist_total', 'properties.sol']]
display(dist_sol_df.head())

hel_dist_sol_df = helicopter_df[['properties.length', 'properties.sol']]
hel_dist_sol_df.loc[:, 'properties.length'] /= 1000
hel_dist_sol_df['properties.dist_total'] = hel_dist_sol_df['properties.length'].cumsum()
hel_dist_sol_df = hel_dist_sol_df.drop('properties.length', axis=1)
hel_dist_sol_df.head()


# In[190]:


# Add starting info
start_df = pd.DataFrame({'properties.dist_total':[0], 'properties.sol':[0]})
dist_sol_df = pd.concat([start_df, dist_sol_df], ignore_index=True)
display(dist_sol_df.head())

hel_dist_sol_df = pd.concat([start_df, hel_dist_sol_df], ignore_index=True)
hel_dist_sol_df


# In[191]:


# Convert from meters to kilometers
if rover_name == 'Perseverance':
    dist_sol_df['properties.dist_total'] /= 1000.0
dist_sol_df.head()


# In[192]:


def color(r, g, b):
    return (r/255.0, g/255.0, b/255.0)

if rover_name == 'Perseverance':
    splits = [
        # Helicopter Splits
        {
            'start_sol':0,
            'end_sol':58,
            'name':'Landed',
            'text_offset_x':10,
            'text_offset_y':-0.07,
            'color':color(50, 50, 100),
            'use_label':False,
            'rover':False,
            'vehicle_name':'Ingenuity'
        },
        {
            'start_sol':58,
            'end_sol':91,
            'name':'Technology Demonstration',
            'text_offset_x':10,
            'text_offset_y':-0.07,
            'color':color(150, 75, 0),
            'use_label':True,
            'rover':False,
            'vehicle_name':'Ingenuity'
        },
        {
            'start_sol':91,
            'end_sol':dist_sol_df['properties.sol'].max(),
            'name':'Operations Demonstration',
            'text_offset_x':-30,
            'text_offset_y':0.10,
            'color':color(255, 215, 0),
            'use_label':True,
            'rover':False,
            'vehicle_name':'Ingenuity'
        },
        # Rover splits
        # Put second so it will be drawn over
        # helicopter, indicating higher priority
        {
            'start_sol':0,
            'end_sol':100,
            'name':'Landed',
            'text_offset_x':10,
            'text_offset_y':-0.07,
            'color':color(5, 100, 152),
            'use_label':True,
            'rover':True,
            'vehicle_name':'Perseverance'
        },
        {
            'start_sol':100,
            'end_sol':dist_sol_df['properties.sol'].max(),
            'name':'First Science Campaign',
            'text_offset_x':10,
            'text_offset_y':-0.07,
            'color':color(199, 72, 84),
            'use_label':True,
            'rover':True,
            'vehicle_name':'Perseverance'
        }
    ]
elif rover_name == 'Curiosity':
    splits = [
        {
            'start_sol':0,
            'end_sol':746,
            'name':'Landed',
            'text_offset_x':180,
            'text_offset_y':3,
            'color':color(3, 25, 252),
            'use_label':False,
            'rover':True,
            'vehicle_name':'Curiosity'
        },
        {
            'start_sol':746,
            'end_sol':2369,
            'name':'Reached Mount Sharp',
            'text_offset_x':-140,
            'text_offset_y':3,
            'color':color(5, 100, 152),
            'use_label':True,
            'rover':True,
            'vehicle_name':'Curiosity'
        },      
        {
            'start_sol':2369,
            'end_sol':dist_sol_df['properties.sol'].max(),
            'name':'Clay Bearing Unit',
            'text_offset_x':-140,
            'text_offset_y':3,
            'color':color(255, 215, 0),
            'use_label':True,
            'rover':True,
            'vehicle_name':'Curiosity'
        }
    ]
    
def get_df_in_split(split, df):
    return df.loc[
        (df['properties.sol'] >= split['start_sol'])
        & (df['properties.sol'] <= split['end_sol'])
    ]


# In[193]:


def get_where_on_sol(sol, df):
    # Get dist_total given sol
    # Chooses latest sol before or equal to input sol
    # i.e if data on sols 1, 3, 7, 10 and input on 9,
    # we return data from 7th sol.
    for idx, row in df.iterrows():
        if row['properties.sol'] > sol:
            return df['properties.dist_total'][idx-1]
        elif row['properties.sol'] == sol:
            return row['properties.dist_total']
        
    # Later than all recorded data, so return latest distance known
    return df['properties.dist_total'].max()


# In[194]:


# Generate graph
plt.style.use('Solarize_Light2')
fig, ax = plt.subplots(1, 1)

# General graph stuff
ax.set_xlabel('Sols on Mars')
ax.set_ylabel('Total Kilometers Traveled')
ax.set_ylim(-0.15, dist_sol_df['properties.dist_total'].max() + 0.1)
ax.set_xlim(0, dist_sol_df['properties.sol'].max())
ax.set_title(f'Distance Traveled by {rover_name}'
             + f" as of Sol {int(dist_sol_df['properties.sol'].max())}")

# For miles conversion
axtwin = ax.twinx()
axtwin.set_ylabel('Total Miles Traveled')
axtwin.set_ylim(-0.15, dist_sol_df['properties.dist_total'].max() + 0.1)
axtwin.grid(None)

# Loop through all splits
for split in splits:
    if split['rover']:
        df = dist_sol_df
    elif use_helicopter:
        df = hel_dist_sol_df
    else:
        continue
        
    in_range_df = get_df_in_split(split, df)
    ax.step(
        in_range_df['properties.sol'],
        in_range_df['properties.dist_total'],
        color=split['color'],
        linestyle='-' if split['rover'] else ':',
        label=split['vehicle_name']
    )
    if split['use_label']:
        ax.annotate(
            split['name'],
            xy=(
                split['start_sol'],
                get_where_on_sol(split['start_sol'], df) - 0.01
            ),
            xytext=(
                split['start_sol'] + split['text_offset_x'],
                get_where_on_sol(split['start_sol'], df)
                + split['text_offset_y']
            ),
            arrowprops={
                'facecolor':'black',
                'headwidth':3,
                'width':0.05,
                'headlength':5
            },
            horizontalalignment='center',
            verticalalignment='top'
        )
    # correct indent level after if statement

# Remove duplicate legend labels
# and show it based on line style not color
handles, labels = ax.get_legend_handles_labels()
by_label = dict(zip(labels, handles))
ax.legend(by_label.values(), by_label.keys())
leg = ax.get_legend()
[lgd.set_color('black') for lgd in leg.legendHandles]
    
    
# Use get_yticks instead of get_yticklabels
# because labels require the plot to be drawn
# first to be generated, and the yticks = yticklabels
# in this case, so it's a valid substitute
axtwin.set_yticklabels(
    [f'{x * 0.62137:0.1f}' for x in ax.get_yticks()]
)

fig.show()
plt.savefig(f'{rover_name} Distance Graph.svg')