Commit 8acb8c3e authored by Qusai Al Shidi's avatar Qusai Al Shidi 💬

Version 2020.1. Changed download_magnetogram_hmi to take a new argument for...

Version 2020.1. Changed download_magnetogram_hmi to take a new argument for either B_720s or b_synoptic_small which is needed.
parent 3a10227e
# Table of Contents
* [swmfpy](#.swmfpy)
* [swmfpy.web](#.swmfpy.web)
* [get\_omni\_data](#.swmfpy.web.get_omni_data)
* [download\_magnetogram\_hmi](#.swmfpy.web.download_magnetogram_hmi)
* [download\_magnetogram\_adapt](#.swmfpy.web.download_magnetogram_adapt)
* [swmfpy.io](#.swmfpy.io)
* [read\_wdc\_ae](#.swmfpy.io.read_wdc_ae)
* [read\_wdc\_asy\_sym](#.swmfpy.io.read_wdc_asy_sym)
* [read\_gm\_log](#.swmfpy.io.read_gm_log)
* [swmfpy.tools](#.swmfpy.tools)
* [swmfpy.paramin](#.swmfpy.paramin)
* [replace\_command](#.swmfpy.paramin.replace_command)
* [read\_command](#.swmfpy.paramin.read_command)
* [swmfpy.web](#.swmfpy.web)
* [get\_omni\_data](#.swmfpy.web.get_omni_data)
* [download\_magnetogram\_hmi](#.swmfpy.web.download_magnetogram_hmi)
* [download\_magnetogram\_adapt](#.swmfpy.web.download_magnetogram_adapt)
<a name=".swmfpy"></a>
## swmfpy
......@@ -35,165 +36,6 @@ These are not automatically imported. Might have extra dependancies.
*None yet.*
<a name=".swmfpy.web"></a>
## swmfpy.web
Tools to retrieve and send data on the web.
Here are a collection of tools to work with data on the internet. Thus,
this module mostly requires an internet connection.
<a name=".swmfpy.web.get_omni_data"></a>
#### get\_omni\_data
```python
get_omni_data(time_from, time_to, **kwargs)
```
Retrieve omni solar wind data over http.
This will download omni data from https://spdf.gsfc.nasa.gov/pub/data/omni
and put it into a dictionary. If your data is large, then make a csv and
use swmfpy.io.read_omni_data().
**Arguments**:
- `time_from` _datetime.datetime_ - The start time of the solar wind
data that you want to receive.
- `time_to` _datetime.datetime_ - The end time of the solar wind data
you want to receive.
**Returns**:
- `dict` - This will be a list of *all* columns
available in the omni data set.
**Examples**:
```python
import datetime
import swmfpy.web
storm_start = datetime.datetime(year=2000, month=1, day=1)
storm_end = datetime.datetime(year=2000, month=2, day=15)
data = swmfpy.web.get_omni_data(time_from=storm_start,
time_to=storm_end)
```
<a name=".swmfpy.web.download_magnetogram_hmi"></a>
#### download\_magnetogram\_hmi
```python
download_magnetogram_hmi(mag_time, **kwargs)
```
Downloads HMI vector magnetogram fits files.
This will download vector magnetogram FITS files from
Joint Science Operations Center (JSOC) near a certain hour.
This unfortunately depends on sunpy/drms, if you don't have it try,
```bash
pip install -U --user drms
```
**Arguments**:
- `mag_time` _datetime.datetime_ - Time after which to find
vector magnetograms.
**kwargs:
- `download_dir` _str_ - Relative directory to download to.
- `verbose` _bool_ - (default False) print out the files it's downloading.
**Returns**:
- `str` - list of filenames downloaded.
**Raises**:
- `ImportError` - If module `drms` is not found.
- `FileNotFoundError` - If the JSOC doesn't have the magnetograms for that
time.
**Examples**:
```python
from swmfpy.web import download_magnetogram_hmi
import datetime as dt
# I am interested in the hmi vector magnetogram from 2014, 2, 18
time_mag = dt.datetime(2014, 2, 18, 10) # Around hour 10
# Calling it will download
filenames = download_magnetogram_hmi(mag_time=time_mag,
download_dir='mydir/')
# To see my list
print('The magnetograms I downloaded are:', filenames)
# You may call and ignore the file list
download_magnetogram_hmi(mag_time=time_mag, download_dir='mydir')
```
<a name=".swmfpy.web.download_magnetogram_adapt"></a>
#### download\_magnetogram\_adapt
```python
download_magnetogram_adapt(time, map_type='fixed', **kwargs)
```
This routine downloads GONG ADAPT magnetograms.
Downloads ADAPT magnetograms from ftp://gong2.nso.edu/adapt/maps/gong/
to a local directory. It will download all maps with the regex file
pattern: adapt4[0,1]3*yyyymmddhh
**Arguments**:
- `time` _datetime.datetime_ - Time in which you want the magnetogram.
- `map_type` _str_ - (default: 'fixed')
Choose either 'fixed' or 'central' for
the map type you want.
**kwargs:
- `download_dir` _str_ - (default is current dir) Relative directory
where you want the maps to be downloaded.
**Returns**:
- `str` - First unzipped filename found.
**Raises**:
- `NotADirectoryError` - If the adapt maps directory
is not found on the server.
- `ValueError` - If map_type is not recognized.
(i.e. not 'fixed' or 'central')
- `FileNotFoundError` - If maps were not found.
**Examples**:
```python
import datetime as dt
# Use datetime objects for the time
time_flare = dt.datetime(2018, 2, 12, hour=10)
swmfpy.web.download_magnetogram_adapt(time=time_flare,
map_type='central',
download_dir='./mymaps/')
```
<a name=".swmfpy.io"></a>
## swmfpy.io
......@@ -300,6 +142,12 @@ from the GM model log.
plt.plot(geo['times', geo['AL'])
```
<a name=".swmfpy.tools"></a>
## swmfpy.tools
Tools to be used in swmfpy functions and classes. Some of the functions are
*hidden functions*.
<a name=".swmfpy.paramin"></a>
## swmfpy.paramin
......@@ -319,7 +167,7 @@ Note, if you have repeat commands this will replace all the repeats.
**Arguments**:
- `parameters` _dict_ - Dictionary of strs with format
replace = {'COMMAND': ['value', 'comments', ...]}
replace = 'COMMAND': ['value', 'comments', ...]
This is case sensitive.
- `input_file` _str_ - String of PARAM.in file name.
- `output_file` _str_ - (default 'PARAM.in') The output file to write to.
......@@ -338,7 +186,7 @@ Note, if you have repeat commands this will replace all the repeats.
**Examples**:
```python
change['SOLARWINDFILE'] = [['T', 'UseSolarWindFile'],
change['`SOLARWINDFILE`'] = [['T', 'UseSolarWindFile'],
['new_imf.dat', 'NameSolarWindFile']]
# This will overwrite PARAM.in
swmfpy.paramin.replace('PARAM.in.template', change)
......@@ -390,3 +238,167 @@ values for the parameters.
this, try using the `num_of_values` keyword. This is helpful if your
PARAM.in is comment heavy.
<a name=".swmfpy.web"></a>
## swmfpy.web
Tools to retrieve and send data on the web.
Here are a collection of tools to work with data on the internet. Thus,
this module mostly requires an internet connection.
<a name=".swmfpy.web.get_omni_data"></a>
#### get\_omni\_data
```python
get_omni_data(time_from, time_to, **kwargs)
```
Retrieve omni solar wind data over http.
This will download omni data from https://spdf.gsfc.nasa.gov/pub/data/omni
and put it into a dictionary. If your data is large, then make a csv and
use swmfpy.io.read_omni_data().
**Arguments**:
- `time_from` _datetime.datetime_ - The start time of the solar wind
data that you want to receive.
- `time_to` _datetime.datetime_ - The end time of the solar wind data
you want to receive.
**Returns**:
- `dict` - This will be a list of *all* columns
available in the omni data set.
**Examples**:
```python
import datetime
import swmfpy.web
storm_start = datetime.datetime(year=2000, month=1, day=1)
storm_end = datetime.datetime(year=2000, month=2, day=15)
data = swmfpy.web.get_omni_data(time_from=storm_start,
time_to=storm_end)
```
<a name=".swmfpy.web.download_magnetogram_hmi"></a>
#### download\_magnetogram\_hmi
```python
download_magnetogram_hmi(mag_time, hmi_map, **kwargs)
```
Downloads HMI vector magnetogram fits files.
This will download vector magnetogram FITS files from
Joint Science Operations Center (JSOC) near a certain hour.
This unfortunately depends on sunpy and drms, if you don't have it try,
```bash
pip install -U --user sunpy drms
```
**Arguments**:
- `mag_time` _datetime.datetime_ - Time after which to find
vector magnetograms.
- `hmi_map` _str_ - JSOC prefix for hmi maps. Currently can only do
'hmi.B_720s' and 'hmi.b_synoptic.small'.
**kwargs:
- `download_dir` _str_ - Relative directory to download to.
- `verbose` _bool_ - (default False) print out the files it's downloading.
**Returns**:
- `str` - list of filenames downloaded.
**Raises**:
- `ImportError` - If module `drms` is not found.
- `FileNotFoundError` - If the JSOC doesn't have the magnetograms for that
time.
**Examples**:
```python
from swmfpy.web import download_magnetogram_hmi
import datetime as dt
# I am interested in the hmi vector magnetogram from 2014, 2, 18
time_mag = dt.datetime(2014, 2, 18, 10) # Around hour 10
# Calling it will download
filenames = download_magnetogram_hmi(mag_time=time_mag,
hmi_map='B_720s',
download_dir='mydir/')
# To see my list
print('The magnetograms I downloaded are:', filenames)
# You may call and ignore the file list
download_magnetogram_hmi(mag_time=time_mag,
hmi_map='b_synoptic_small',
download_dir='mydir')
```
<a name=".swmfpy.web.download_magnetogram_adapt"></a>
#### download\_magnetogram\_adapt
```python
download_magnetogram_adapt(time, map_type='fixed', **kwargs)
```
This routine downloads GONG ADAPT magnetograms.
Downloads ADAPT magnetograms from ftp://gong2.nso.edu/adapt/maps/gong/
to a local directory. It will download all maps with the regex file
pattern: adapt4[0,1]3*yyyymmddhh
**Arguments**:
- `time` _datetime.datetime_ - Time in which you want the magnetogram.
- `map_type` _str_ - (default: 'fixed')
Choose either 'fixed' or 'central' for
the map type you want.
**kwargs:
- `download_dir` _str_ - (default is current dir) Relative directory
where you want the maps to be downloaded.
**Returns**:
- `str` - First unzipped filename found.
**Raises**:
- `NotADirectoryError` - If the adapt maps directory
is not found on the server.
- `ValueError` - If map_type is not recognized.
(i.e. not 'fixed' or 'central')
- `FileNotFoundError` - If maps were not found.
**Examples**:
```python
import datetime as dt
# Use datetime objects for the time
time_flare = dt.datetime(2018, 2, 12, hour=10)
swmfpy.web.download_magnetogram_adapt(time=time_flare,
map_type='central',
download_dir='./mymaps/')
```
......@@ -10,7 +10,7 @@ with open('README.markdown') as readme_fh:
setuptools.setup(
name='swmfpy',
version='2020.0',
version='2020.1',
author='Qusai Al Shidi',
author_email='qusai@umich.edu',
description='''A collection of tools for the Space Weather Modelling
......
......@@ -19,7 +19,7 @@ These are not automatically imported. Might have extra dependancies.
"""
__author__ = 'Qusai Al Shidi'
__license__ = 'MIT'
__version__ = '2020.0'
__version__ = '2020.1'
__maintainer__ = 'Qusai Al Shidi'
__email__ = 'qusai@umich.edu'
......
......@@ -17,7 +17,7 @@ def replace_command(parameters, input_file, output_file='PARAM.in'):
Args:
parameters (dict): Dictionary of strs with format
replace = {'COMMAND': ['value', 'comments', ...]}
replace = 'COMMAND': ['value', 'comments', ...]
This is case sensitive.
input_file (str): String of PARAM.in file name.
output_file (str): (default 'PARAM.in') The output file to write to.
......@@ -30,7 +30,7 @@ def replace_command(parameters, input_file, output_file='PARAM.in'):
Examples:
```python
change['SOLARWINDFILE'] = [['T', 'UseSolarWindFile'],
change['#SOLARWINDFILE'] = [['T', 'UseSolarWindFile'],
['new_imf.dat', 'NameSolarWindFile']]
# This will overwrite PARAM.in
swmfpy.paramin.replace('PARAM.in.template', change)
......
#!/usr/bin/env python
"""Tools to be used in swmfpy functions and classes. Some of the functions are
*hidden functions*.
"""
__author__ = 'Qusai Al Shidi'
__email__ = 'qusai@umich.edu'
def _import_error_string(library):
return f'Error importing drms. Maybe try `pip install -U {library} --user ` .'
def _nearest(pivot, items):
"""Returns nearest point"""
return min(items, key=lambda x: abs(x - pivot))
......@@ -8,6 +8,7 @@ __email__ = 'qusai@umich.edu'
import datetime as dt
import urllib
from .tools import _import_error_string, _nearest
def get_omni_data(time_from, time_to, **kwargs):
......@@ -142,21 +143,24 @@ def _check_bad_omni_num(value_string):
return True
def download_magnetogram_hmi(mag_time, **kwargs):
def download_magnetogram_hmi(mag_time, hmi_map='hmi.B_720s', **kwargs):
"""Downloads HMI vector magnetogram fits files.
This will download vector magnetogram FITS files from
Joint Science Operations Center (JSOC) near a certain hour.
This unfortunately depends on sunpy/drms, if you don't have it try,
This unfortunately depends on sunpy and drms, if you don't have it try,
```bash
pip install -U --user drms
pip install -U --user sunpy drms
```
Args:
mag_time (datetime.datetime): Time after which to find
vector magnetograms.
hmi_map (str): JSOC prefix for hmi maps. Currently can only do
'hmi.B_720s' and 'hmi.b_synoptic.small'.
**kwargs:
download_dir (str): Relative directory to download to.
......@@ -180,13 +184,16 @@ def download_magnetogram_hmi(mag_time, **kwargs):
# Calling it will download
filenames = download_magnetogram_hmi(mag_time=time_mag,
hmi_map='B_720s',
download_dir='mydir/')
# To see my list
print('The magnetograms I downloaded are:', filenames)
# You may call and ignore the file list
download_magnetogram_hmi(mag_time=time_mag, download_dir='mydir')
download_magnetogram_hmi(mag_time=time_mag,
hmi_map='b_synoptic_small',
download_dir='mydir')
```
"""
......@@ -194,24 +201,15 @@ def download_magnetogram_hmi(mag_time, **kwargs):
try:
import drms
except ImportError:
raise ImportError('''Error importing drms. Maybe try
`pip install -U --user drms` .
''')
raise ImportError(_import_error_string('drms'))
get_urls = {
'hmi.B_720s': _get_urls_hmi_b720,
'hmi.b_synoptic_small': _get_urls_hmi_b_synoptic_small,
}
client = drms.Client()
query_string = 'hmi.B_720s['
query_string += f'{mag_time.year}.'
query_string += f'{str(mag_time.month).zfill(2)}.'
query_string += f'{str(mag_time.day).zfill(2)}_'
query_string += f'{str(mag_time.hour).zfill(2)}'
query_string += f'/1h]'
data = client.query(query_string, key='T_REC', seg='field')
times = drms.to_datetime(data[0].T_REC)
nearest_time = _nearest(mag_time, times)
# Generator to find the nearest time
urls = ((data_time, mag_url) for (data_time, mag_url)
in zip(times, data[1].field) if data_time == nearest_time)
urls = get_urls[hmi_map](client, mag_time)
# Download data
if kwargs.get('verbose', False):
......@@ -241,6 +239,61 @@ def download_magnetogram_hmi(mag_time, **kwargs):
return return_name
def _get_urls_hmi_b_synoptic_small(client, mag_time):
"""Returns for #download_magnetogram_hmi needed urls
Args:
client (drms.Client): To query and return urls.
mag_time (datetime.datetime): To find nearest magnetogram.
Returns:
generator that yields (datetime.datetime, str): Time of magnetogram,
suffix url of magnetogram
"""
import drms
try:
from sunpy.coordinates.sun import carrington_rotation_number
except ImportError as error:
print(_import_error_string('sunpy'))
raise error
cr_number = int(round(carrington_rotation_number(mag_time)))
query_string = f'hmi.b_synoptic_small[{int(round(cr_number))}]'
components = ['Bp', 'Bt', 'Br']
data = client.query(query_string, seg=components)
# Generator to find the nearest time
prefix_str = 'CR' + str(cr_number) + '_' + str(mag_time)
urls = ((prefix_str, data[component][0]) for component in components)
return urls
def _get_urls_hmi_b720(client, mag_time):
"""Returns for #download_magnetogram_hmi needed urls for hmi.B_720s
Args:
client (drms.Client): To query and return urls.
mag_time (datetime.datetime): To find nearest magnetogram.
Returns:
generator that yields (datetime.datetime, str): Time of magnetogram,
suffix url of magnetogram
"""
import drms
query_string = 'hmi.B_720s'
query_string += f'[{mag_time.year}.'
query_string += f'{str(mag_time.month).zfill(2)}.'
query_string += f'{str(mag_time.day).zfill(2)}_'
query_string += f'{str(mag_time.hour).zfill(2)}'
query_string += f'/1h]'
data = client.query(query_string, key='T_REC', seg='field')
times = drms.to_datetime(data[0].T_REC)
nearest_time = _nearest(mag_time, times)
# Generator to find the nearest time
urls = ((data_time, mag_url) for (data_time, mag_url)
in zip(times, data[1].field) if data_time == nearest_time)
return urls
def download_magnetogram_adapt(time, map_type='fixed', **kwargs):
"""This routine downloads GONG ADAPT magnetograms.
......@@ -356,8 +409,3 @@ def download_magnetogram_adapt(time, map_type='fixed', **kwargs):
if '.gz' in filename:
return_names[index] = filename.replace('.gz', '')
return return_names
def _nearest(pivot, items):
"""Returns nearest point"""
return min(items, key=lambda x: abs(x - pivot))
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment