Note: The default ITS GitLab runner is a shared resource and is subject to slowdowns during heavy usage.
You can run your own GitLab runner that is dedicated just to your group if you need to avoid processing delays.

Commit 48200d70 authored by Qusai Al Shidi's avatar Qusai Al Shidi 💬
Browse files

Merge branch 'imf-f107' into 'master'

Imf f107

See merge request swmf_software/swmfpy!2
parents d4a5c35f 3c757873
# Table of Contents # Table of Contents
* [swmfpy](#.swmfpy) * [swmfpy](#.swmfpy)
* [swmfpy.tools](#.swmfpy.tools) * [write\_imf\_from\_omni](#.swmfpy.write_imf_from_omni)
* [swmfpy.io](#.swmfpy.io)
* [read\_wdc\_ae](#.swmfpy.io.read_wdc_ae)
* [read\_wdc\_asy\_sym](#.swmfpy.io.read_wdc_asy_sym)
* [read\_gm\_log](#.swmfpy.io.read_gm_log)
* [swmfpy.web](#.swmfpy.web) * [swmfpy.web](#.swmfpy.web)
* [get\_omni\_data](#.swmfpy.web.get_omni_data) * [get\_omni\_data](#.swmfpy.web.get_omni_data)
* [download\_magnetogram\_hmi](#.swmfpy.web.download_magnetogram_hmi) * [download\_magnetogram\_hmi](#.swmfpy.web.download_magnetogram_hmi)
* [download\_magnetogram\_adapt](#.swmfpy.web.download_magnetogram_adapt) * [download\_magnetogram\_adapt](#.swmfpy.web.download_magnetogram_adapt)
* [swmfpy.io](#.swmfpy.io)
* [write\_imf\_input](#.swmfpy.io.write_imf_input)
* [read\_wdc\_ae](#.swmfpy.io.read_wdc_ae)
* [read\_wdc\_asy\_sym](#.swmfpy.io.read_wdc_asy_sym)
* [read\_gm\_log](#.swmfpy.io.read_gm_log)
* [swmfpy.paramin](#.swmfpy.paramin) * [swmfpy.paramin](#.swmfpy.paramin)
* [replace\_command](#.swmfpy.paramin.replace_command) * [replace\_command](#.swmfpy.paramin.replace_command)
* [read\_command](#.swmfpy.paramin.read_command) * [read\_command](#.swmfpy.paramin.read_command)
* [swmfpy.tools](#.swmfpy.tools)
<a name=".swmfpy"></a> <a name=".swmfpy"></a>
## swmfpy ## swmfpy
...@@ -34,114 +36,36 @@ These are not automatically imported. Might have extra dependancies. ...@@ -34,114 +36,36 @@ These are not automatically imported. Might have extra dependancies.
*None yet.* *None yet.*
<a name=".swmfpy.tools"></a> <a name=".swmfpy.write_imf_from_omni"></a>
## swmfpy.tools #### write\_imf\_from\_omni
Tools to be used in swmfpy functions and classes. Some of the functions are
*hidden functions*.
<a name=".swmfpy.io"></a>
## swmfpy.io
### Input/Output tools
Here are tools to read and write files relating to SWMF.
<a name=".swmfpy.io.read_wdc_ae"></a>
#### read\_wdc\_ae
```python ```python
read_wdc_ae(wdc_filename) write_imf_from_omni(time_from, time_to, filename='IMF.dat', **kwargs)
``` ```
Read an auroral electrojet (AE) indeces from Kyoto's World Data Center Writes an IMF.dat file for the geospace model runs for a specific time
text file into a dictionary of lists. period.
**Arguments**: **Arguments**:
- `time_from` _datetime.datetime_ - Time to begin omni data retrieval
- `wdc_filename` _str_ - Filename of wdc data from - `time_to` _datetime.datetime_ - Time to end omni data retrieval
http://wdc.kugi.kyoto-u.ac.jp/ - `filename` _str_ - The filename for the dat file, defaults to 'IMF.dat'.
**kwargs:
**Returns**: see `swmfpy.io.write_imf_input()` and `swmfpy.web.get_omni_data()`
- `dict` - Auroral indeces 'AL', 'AE', 'AO', 'AU'
- `datetime.datetime` - 'times'
- `int` - 'values'
<a name=".swmfpy.io.read_wdc_asy_sym"></a>
#### read\_wdc\_asy\_sym
```python
read_wdc_asy_sym(wdc_filename)
```
Reads a WDC file for ASY/SYM data.
Reads an ASY/SYM file downloaded from
http://wdc.kugi.kyoto-u.ac.jp/aeasy/index.html
and puts it into a dictionary.
**Arguments**:
- `wdc_filename` _str_ - Relative filename (or file handle no.) to read.
**Returns**:
- `dict` - of values. {'[ASY-SYM]-[D-H]': 'times': [], 'values': []}
**Examples**:
```python
indeces = swmfpy.io.read_wdc_asy_sym('wdc.dat')
# Plot data
plt.plot(indeces['SYM-H']['times'],
indeces['SYM-H']['values'],
label='SYM-H [nT]'
)
plt.xlabel('Time [UT]')
```
Important to note if there is bad data it will be filled as None.
<a name=".swmfpy.io.read_gm_log"></a>
#### read\_gm\_log
```python
read_gm_log(filename, colnames=None, dtypes=None, index_time=True)
```
Make a dictionary out of the indeces outputted
from the GM model log.
**Arguments**:
- `filename` _str_ - The relative filename as a string. (or file handle no.)
- `colnames` _[str]_ - (default: None) Supply the name of the columns.
If None it will use second line
of log file.
- `dtypes` _[types]_ - (default: None) Provide types for the columns, if
None then all will be float.
- `index_time` _bool_ - (default: True) Make a column of dt.datetime objects
in dictionary key 'Time [UT]'.
**Returns**:
- `dict` - Dictionary of the log file
**Examples**: **Examples**:
To plot AL and Dst get the log files Using this function is simple:
```python ```python
geo = swmfpy.io.read_gm_log('run/GM/IO2/geoindex_e20140215-100500.log') import swmfpy
dst = swmfpy.io.read_gm_log('run/GM/IO2/log_e20140215-100500.log') import datetime as dt
times = (dt.datetime(2014, 2, 2), dt.datetime(2014, 2, 4))
# Plot AL indeces # Usually the kwargs are unecessary
plt.plot(geo['times', geo['AL']) swmfpy.write_imf_input(*times)
# Sometimes this
swmfpy.write_imf_input(*times, filename='run/IMF.dat')
``` ```
<a name=".swmfpy.web"></a> <a name=".swmfpy.web"></a>
...@@ -173,6 +97,14 @@ use swmfpy.io.read_omni_data(). ...@@ -173,6 +97,14 @@ use swmfpy.io.read_omni_data().
data that you want to receive. data that you want to receive.
- `time_to` _datetime.datetime_ - The end time of the solar wind data - `time_to` _datetime.datetime_ - The end time of the solar wind data
you want to receive. you want to receive.
**kwargs:
- `original_colnames` _bool_ - Use the original column names from the
spdf specification. The alternative is
nicer and shorter names. Defaults to
False.
- `resolution` _str_ - (default: 'high') Here you can choose 'high' or
'low' resolution omni data. Some columns appear
in one but not the other.
**Returns**: **Returns**:
...@@ -191,6 +123,10 @@ use swmfpy.io.read_omni_data(). ...@@ -191,6 +123,10 @@ use swmfpy.io.read_omni_data().
storm_end = datetime.datetime(year=2000, month=2, day=15) storm_end = datetime.datetime(year=2000, month=2, day=15)
data = swmfpy.web.get_omni_data(time_from=storm_start, data = swmfpy.web.get_omni_data(time_from=storm_start,
time_to=storm_end) time_to=storm_end)
# or for low res
data = swmfpy.web.get_omni_data(time_from=storm_start,
time_to=storm_end,
resolution='low')
``` ```
<a name=".swmfpy.web.download_magnetogram_hmi"></a> <a name=".swmfpy.web.download_magnetogram_hmi"></a>
...@@ -310,6 +246,151 @@ pattern: adapt4[0,1]3*yyyymmddhh ...@@ -310,6 +246,151 @@ pattern: adapt4[0,1]3*yyyymmddhh
download_dir='./mymaps/') download_dir='./mymaps/')
``` ```
<a name=".swmfpy.io"></a>
## swmfpy.io
### Input/Output tools
Here are tools to read and write files relating to SWMF.
<a name=".swmfpy.io.write_imf_input"></a>
#### write\_imf\_input
```python
write_imf_input(imf_data, filename='IMF.dat', **kwargs)
```
Creates the IMF.dat input file for the SWMF BATS-R-US geospace model.
`imf_data` must be a dictionary of array_like objects with same length
in data. In swmfpy Pythonic versions are always preferred so the 'times'
must be `datetime.datetime` array.
imf_data = dict(times, bx, by, bz, vx, vy, vz, density, temperature)
**Arguments**:
- `imf_data` _dict_ - This dictionary contains the solar wind data.
- `filename` _str_ - (default: 'IMF.dat') Filename to write to.
**kwargs:
- `commands` _[str]_ - (default: None) List of commands to write to imf
input file (indexed by line then by tabs on same
line). *Note*: Must be a list if have one command
str.
**Raises**:
- `TypeError` - If commands is not a list or tuple. It must be at least a
one element list of strings.
- `AssertionError` - If inputs aren't prepared properly (key names)
**Examples**:
```python
from swmfpy.io import write_imf_input
# Prepare imf dictionary: imf_data
write_imf_input(imf_data, filename='run/IMF.dat')
```
<a name=".swmfpy.io.read_wdc_ae"></a>
#### read\_wdc\_ae
```python
read_wdc_ae(wdc_filename)
```
Read an auroral electrojet (AE) indeces from Kyoto's World Data Center
text file into a dictionary of lists.
**Arguments**:
- `wdc_filename` _str_ - Filename of wdc data from
http://wdc.kugi.kyoto-u.ac.jp/
**Returns**:
- `dict` - Auroral indeces 'AL', 'AE', 'AO', 'AU'
- `datetime.datetime` - 'times'
- `int` - 'values'
<a name=".swmfpy.io.read_wdc_asy_sym"></a>
#### read\_wdc\_asy\_sym
```python
read_wdc_asy_sym(wdc_filename)
```
Reads a WDC file for ASY/SYM data.
Reads an ASY/SYM file downloaded from
http://wdc.kugi.kyoto-u.ac.jp/aeasy/index.html
and puts it into a dictionary.
**Arguments**:
- `wdc_filename` _str_ - Relative filename (or file handle no.) to read.
**Returns**:
- `dict` - of values. {'[ASY-SYM]-[D-H]': 'times': [], 'values': []}
**Examples**:
```python
indeces = swmfpy.io.read_wdc_asy_sym('wdc.dat')
# Plot data
plt.plot(indeces['SYM-H']['times'],
indeces['SYM-H']['values'],
label='SYM-H [nT]'
)
plt.xlabel('Time [UT]')
```
Important to note if there is bad data it will be filled as None.
<a name=".swmfpy.io.read_gm_log"></a>
#### read\_gm\_log
```python
read_gm_log(filename, colnames=None, dtypes=None, index_time=True)
```
Make a dictionary out of the indeces outputted
from the GM model log.
**Arguments**:
- `filename` _str_ - The relative filename as a string. (or file handle no.)
- `colnames` _[str]_ - (default: None) Supply the name of the columns.
If None it will use second line
of log file.
- `dtypes` _[types]_ - (default: None) Provide types for the columns, if
None then all will be float.
- `index_time` _bool_ - (default: True) Make a column of dt.datetime objects
in dictionary key 'Time [UT]'.
**Returns**:
- `dict` - Dictionary of the log file
**Examples**:
To plot AL and Dst get the log files
```python
geo = swmfpy.io.read_gm_log('run/GM/IO2/geoindex_e20140215-100500.log')
dst = swmfpy.io.read_gm_log('run/GM/IO2/log_e20140215-100500.log')
# Plot AL indeces
plt.plot(geo['times', geo['AL'])
```
<a name=".swmfpy.paramin"></a> <a name=".swmfpy.paramin"></a>
## swmfpy.paramin ## swmfpy.paramin
...@@ -331,7 +412,7 @@ Note, if you have repeat commands this will replace all the repeats. ...@@ -331,7 +412,7 @@ Note, if you have repeat commands this will replace all the repeats.
**Arguments**: **Arguments**:
- `parameters` _dict_ - Dictionary of strs with format - `parameters` _dict_ - Dictionary of strs with format
replace = '#COMMAND': ['value', 'comments', ...] replace = '\#COMMAND': ['value', 'comments', ...]
This is case sensitive. This is case sensitive.
- `input_file` _str_ - String of PARAM.in file name. - `input_file` _str_ - String of PARAM.in file name.
- `output_file` _str_ - (default 'PARAM.in') The output file to write to. - `output_file` _str_ - (default 'PARAM.in') The output file to write to.
...@@ -350,7 +431,7 @@ Note, if you have repeat commands this will replace all the repeats. ...@@ -350,7 +431,7 @@ Note, if you have repeat commands this will replace all the repeats.
**Examples**: **Examples**:
```python ```python
change['#SOLARWINDFILE'] = [['T', 'UseSolarWindFile'], change['\#SOLARWINDFILE'] = [['T', 'UseSolarWindFile'],
['new_imf.dat', 'NameSolarWindFile']] ['new_imf.dat', 'NameSolarWindFile']]
# This will overwrite PARAM.in # This will overwrite PARAM.in
swmfpy.paramin.replace('PARAM.in.template', change) swmfpy.paramin.replace('PARAM.in.template', change)
...@@ -365,12 +446,12 @@ read_command(command, paramin_file='PARAM.in', **kwargs) ...@@ -365,12 +446,12 @@ read_command(command, paramin_file='PARAM.in', **kwargs)
Get parameters of a certain command in PARAM.in file. Get parameters of a certain command in PARAM.in file.
This will find the #COMMAND and return a list of This will find the \#COMMAND and return a list of
values for the parameters. values for the parameters.
**Arguments**: **Arguments**:
- `command` _str_ - This is the #COMMAND you're looking for. - `command` _str_ - This is the \#COMMAND you're looking for.
- `paramin_file` _str_ - (default: 'PARAM.in') The file in which you're - `paramin_file` _str_ - (default: 'PARAM.in') The file in which you're
looking for the command values. looking for the command values.
**kwargs: **kwargs:
...@@ -380,20 +461,20 @@ values for the parameters. ...@@ -380,20 +461,20 @@ values for the parameters.
**Returns**: **Returns**:
- `list` - Values found for the #COMMAND in file. Index 0 is - `list` - Values found for the \#COMMAND in file. Index 0 is
#COMMAND and the values follow (1 for first argument...) \#COMMAND and the values follow (1 for first argument...)
**Raises**: **Raises**:
- `ValueError` - When the #COMMAND is not found. - `ValueError` - When the \#COMMAND is not found.
**Examples**: **Examples**:
```python ```python
start_time = swmfpy.paramin.read_command('#STARTTIME') start_time = swmfpy.paramin.read_command('\#STARTTIME')
end_time = swmfpy.paramin.read_command('#ENDTIME') end_time = swmfpy.paramin.read_command('\#ENDTIME')
print('Starting month is ', start_time[1]) print('Starting month is ', start_time[1])
print('Ending month is ', end_time[1]) print('Ending month is ', end_time[1])
``` ```
...@@ -402,3 +483,9 @@ values for the parameters. ...@@ -402,3 +483,9 @@ values for the parameters.
this, try using the `num_of_values` keyword. This is helpful if your this, try using the `num_of_values` keyword. This is helpful if your
PARAM.in is comment heavy. PARAM.in is comment heavy.
<a name=".swmfpy.tools"></a>
## swmfpy.tools
Tools to be used in swmfpy functions and classes. Some of the functions are
*hidden functions*.
...@@ -15,17 +15,19 @@ Clone into the directory you want to use it. ...@@ -15,17 +15,19 @@ Clone into the directory you want to use it.
*Note*: swmfpy also is part of the SWMF and gets cloned into `SWMF/share/Python`. However, if you would like to [develop](CONTRIBUTING.markdown) for swmfpy or have a local copy do the following: *Note*: swmfpy also is part of the SWMF and gets cloned into `SWMF/share/Python`. However, if you would like to [develop](CONTRIBUTING.markdown) for swmfpy or have a local copy do the following:
```bash ```bash
# Skip this if using it in SWMF directory.
git clone https://gitlab.umich.edu/swmf_sofware/swmfpy.git /path/to/my/dir git clone https://gitlab.umich.edu/swmf_sofware/swmfpy.git /path/to/my/dir
``` ```
Then go to its directory and run `setup.py` make sure to include `--user`. Then go to its directory and run [pip](https://pip.pypa.io/en/stable/) to install. Make sure to include `--user`.
```bash ```bash
cd /path/to/swmfpy cd SWMF/share/Python # or your clone directory
python3 -m pip install setuptools wheel twine --user # If you don't have these pip install --user .
python3 setup.py install --user
``` ```
*Note*: Depending on your system [pip](https://pip.pypa.io/en/stable/) may be ran in several ways: `pip`, `pip3`, or `python3 -m pip`
Then import it into your python project. Then import it into your python project.
```python ```python
......
...@@ -5,8 +5,11 @@ swmfpy. ...@@ -5,8 +5,11 @@ swmfpy.
import setuptools import setuptools
with open('README.markdown') as readme_fh: with open('README.markdown') as fh_readme:
LONG_DESCRIPTION = readme_fh.read() LONG_DESCRIPTION = fh_readme.read()
with open('requirements.txt') as fh_requirements:
REQUIREMENTS = list(fh_requirements)
setuptools.setup( setuptools.setup(
name='swmfpy', name='swmfpy',
...@@ -24,5 +27,6 @@ setuptools.setup( ...@@ -24,5 +27,6 @@ setuptools.setup(
'License :: OSI Approved :: MIT License', 'License :: OSI Approved :: MIT License',
'Operating System :: OS Independent', 'Operating System :: OS Independent',
], ],
install_requires=REQUIREMENTS,
python_requires='>=3.6' python_requires='>=3.6'
) )
...@@ -4,5 +4,5 @@ ...@@ -4,5 +4,5 @@
# run `pip install pydoc-markdown --user` # run `pip install pydoc-markdown --user`
# then run this script in the project root directory # then run this script in the project root directory
pydoc-markdown -v -I. --render-toc -m swmfpy -m swmfpy.web -m swmfpy.io -m swmfpy.paramin -m swmfpy.tools | sed 's/\\043/#/g' - > DOCUMENTATION.markdown pydoc-markdown --py3 -v -I. --render-toc -m swmfpy -m swmfpy.web -m swmfpy.io -m swmfpy.paramin -m swmfpy.tools | sed 's/\\043/#/g' - > DOCUMENTATION.markdown
...@@ -23,51 +23,42 @@ __email__ = 'qusai@umich.edu' ...@@ -23,51 +23,42 @@ __email__ = 'qusai@umich.edu'
import sys import sys
assert sys.version_info >= (3, 6), "swmfpy requires Python >=3.6. Sorry :(."
from . import paramin from . import paramin
from . import io from . import io
from . import web from . import web
assert sys.version_info >= (3, 6), "swmfpy requires Python >=3.6. Sorry :(."
def write_imf_from_omni(time_from, time_to, filename='IMF.dat', **kwargs):
"""Writes an IMF.dat file for the geospace model runs for a specific time
period.
Args:
time_from (datetime.datetime): Time to begin omni data retrieval
time_to (datetime.datetime): Time to end omni data retrieval
filename (str): The filename for the dat file, defaults to 'IMF.dat'.
**kwargs:
see #swmfpy.io.write_imf_input() and #swmfpy.web.get_omni_data()
# This is straight from the format guide on spdf Examples:
OMNI_COLS = ('ID for IMF spacecraft', Using this function is simple:
'ID for SW Plasma spacecraft', ```python
'# of points in IMF averages', import swmfpy
'# of points in Plasma averages', import datetime as dt
'Percent interp', times = (dt.datetime(2014, 2, 2), dt.datetime(2014, 2, 4))
'Timeshift, sec', # Usually the kwargs are unecessary
'RMS, Timeshift', swmfpy.write_imf_input(*times)
'RMS, Phase front normal', # Sometimes this
'Time btwn observations, sec', swmfpy.write_imf_input(*times, filename='run/IMF.dat')
'Field magnitude average, nT', ```
'Bx, nT (GSE, GSM)', """
'By, nT (GSE)', omni_data = web.get_omni_data(time_from, time_to, **kwargs)
'Bz, nT (GSE)', commands = ['#COOR', 'GSE']
'By, nT (GSM)', if kwargs.get('commands', None):
'Bz, nT (GSM)', kwargs['commands'] += commands
'RMS SD B scalar, nT', else:
'RMS SD field vector, nT', kwargs['commands'] = commands
'Flow speed, km/s', column_keys = ['times',
'Vx Velocity, km/s, GSE', 'bx', 'by_gse', 'bz_gse', 'vx_gse', 'vy_gse', 'vz_gse',
'Vy Velocity, km/s, GSE', 'density', 'temperature']
'Vz Velocity, km/s, GSE', io.write_imf_input(omni_data, filename, column_keys=column_keys, **kwargs)
'Proton Density, n/cc',
'Temperature, K',
'Flow pressure, nPa',
'Electric field, mV/m',
'Plasma beta',
'Alfven mach number',
'X(s/c), GSE, Re',
'Y(s/c), GSE, Re',
'Z(s/c), GSE, Re',
'BSN location, Xgse, Re',
'BSN location, Ygse, Re',
'BSN location, Zgse, Re',
'AE-index, nT',
'AL-index, nT',
'AU-index, nT',
'SYM/D index, nT',
'SYM/H index, nT',
'ASY/D index, nT',
'ASY/H index, nT',
'PC(N) index',
'Magnetosonic mach number')