Note: The default ITS GitLab runner is a shared resource and is subject to slowdowns during heavy usage.
You can run your own GitLab runner that is dedicated just to your group if you need to avoid processing delays.

Verified Commit fbd63523 authored by Qusai Al Shidi's avatar Qusai Al Shidi 💬
Browse files

MAINT: Rename download webpage function for reusability.

parent f2c47aec
Pipeline #32431 passed with stage
in 57 seconds
...@@ -214,7 +214,7 @@ def get_omni_data(time_from, time_to, **kwargs): ...@@ -214,7 +214,7 @@ def get_omni_data(time_from, time_to, **kwargs):
for url in omni['urls'](time_from, time_to): for url in omni['urls'](time_from, time_to):
# Parse omni data # Parse omni data
for line in _download_omni_data(url): for line in _download_static_page(url):
cols = line.decode('ascii').split() cols = line.decode('ascii').split()
time = omni['parsetime'](cols) time = omni['parsetime'](cols)
...@@ -235,8 +235,16 @@ def get_omni_data(time_from, time_to, **kwargs): ...@@ -235,8 +235,16 @@ def get_omni_data(time_from, time_to, **kwargs):
@filecache @filecache
def _download_omni_data(url): def _download_static_page(url):
"""Downloads omni data and returns as list """Downloads a cached webpage as a list of lines
Args:
url (str):
Webapage URL.
Returns:
list: A list of lines from the webpage. *Note*: Linebreaks are still
present.
""" """
return list(urllib.request.urlopen(url)) return list(urllib.request.urlopen(url))
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment