icaoberg / python-get-forecast

Created Sun, 26 Apr 2026 00:00:00 +0000 Modified Mon, 27 Apr 2026 21:34:25 -0400
1152 Words
github-actions python parquet weather national-weather-service automation pittsburgh

python-get-forecast is a small repository with a single job: every night at midnight UTC, fetch Pittsburgh’s nightly weather forecast from the National Weather Service, append it to a growing Parquet archive, regenerate the README, and commit everything back to the repository. It has been running since 2022 and has over 1,300 successful workflow runs.

The repo is less about the weather data and more about a pattern: using a scheduled GitHub Action as a lightweight, serverless cron replacement that writes back to its own repository.

What is a GitHub Action?

GitHub Actions is GitHub’s built-in automation platform. You define workflows in YAML files under .github/workflows/, and GitHub runs them in response to events — a push, a pull request, a release, a manual button click, or a schedule.

Each workflow runs inside a fresh virtual machine (called a runner) hosted by GitHub. The runner checks out your repository, executes the steps you define, and then disappears. You pay nothing for public repositories up to the free tier limits, and you don’t manage any infrastructure.

Workflows are triggered by events. The most common for CI/CD are push and pull_request. But there are two others worth knowing:

  • workflow_dispatch — a manual trigger; adds a “Run workflow” button in the GitHub UI
  • schedule — a cron expression; runs the workflow on a timer, independent of any code push

The schedule trigger is what makes python-get-forecast work. The workflow file declares:

on:
  workflow_dispatch:
  schedule:
    - cron: "0 0 * * *"

0 0 * * * means “at minute 0 of hour 0, every day” — midnight UTC. GitHub fires the workflow on that schedule whether or not anyone has touched the repository. The workflow_dispatch entry adds the manual trigger as a fallback for testing or one-off runs.

What the workflow does

The full workflow is straightforward:

name: Data build

on:
  workflow_dispatch:
  schedule:
    - cron: "0 0 * * *"

jobs:
  update-resources:
    name: Update resources
    runs-on: ubuntu-latest
    steps:
      - name: Checkout repository
        uses: actions/checkout@v4
      - name: Setup Python
        uses: actions/setup-python@v5
        with:
          python-version: '3.10'
          cache: 'pip'
      - name: Install dependencies
        run: pip install -r requirements.txt
      - name: Run data build script
        run: python script.py
      - name: Commit and push changes
        run: |
          git config user.name "github-actions[bot]"
          git config user.email "github-actions[bot]@users.noreply.github.com"
          git add README.md weather.parquet
          if git diff --cached --quiet; then
            echo "No changes to commit."
          else
            git commit -m "Update forecast data [skip ci]"
            git push
          fi

Five steps:

  1. Checkout — clone the current repository into the runner
  2. Setup Python — install Python 3.10 with pip caching so dependencies don’t re-download every run
  3. Install dependencies — pip install -r requirements.txt (geopy, requests, pandas, pyarrow)
  4. Run the script — fetch the forecast, update the Parquet file, rewrite the README
  5. Commit and push — stage the two changed files, check whether there’s actually a diff, and push if so

Two details in the commit step are worth noting. The if git diff --cached --quiet check makes the commit conditional: if the forecast hasn’t changed (the API returned identical data), nothing is committed. And the [skip ci] tag in the commit message tells GitHub Actions not to re-trigger the workflow on its own commit — without it, you’d have an infinite loop.

What the script does

The Python script is the core of the repo. It does four things:

1. Geocode the city. The National Weather Service API is point-based — you give it coordinates, it returns a forecast grid URL. The script uses geopy with the Nominatim geocoder to turn "Pittsburgh" into a latitude/longitude pair.

2. Fetch the nightly forecast. Two NWS API calls: first to api.weather.gov/points/{lat},{lon} to get the forecast URL for that grid point, then to that forecast URL to get the list of forecast periods. The script looks for the period named "Tonight" and returns it.

def get_forecast(city=CITY):
    geolocator = Nominatim(user_agent="ModernProgramming")
    location = geolocator.geocode(city)

    url = f"https://api.weather.gov/points/{location.latitude},{location.longitude}"
    response = requests.get(url, timeout=10)
    forecast_url = response.json()["properties"]["forecast"]

    response = requests.get(forecast_url, timeout=10)
    periods = response.json()["properties"]["periods"]

    for period in periods:
        if period["name"] == "Tonight":
            return period

3. Append to the Parquet archive. If weather.parquet exists, it’s loaded into a DataFrame. The new record — start time, end time, and the detailed forecast text — is appended, duplicates are dropped, and the file is written back with zstd compression.

if Path(DATA_FILE).exists():
    df = pd.read_parquet(DATA_FILE)
else:
    df = pd.DataFrame(columns=["Start Date", "End Date", "Forecast"])

datum = {
    "Start Date": period["startTime"],
    "End Date": period["endTime"],
    "Forecast": period["detailedForecast"],
}

df = pd.concat([df, pd.DataFrame([datum])], ignore_index=True)
df = df.drop_duplicates()
df.to_parquet(DATA_FILE, compression="zstd")

4. Rewrite the README. The full historical DataFrame is rendered as a GitHub-flavored Markdown table via df.to_markdown(tablefmt="github") and written to README.md. The README is the human-readable view of the archive — open the repo on any day and you see the complete forecast history.

Why Parquet for this

Storing nightly forecasts as Parquet rather than JSON, CSV, or a plain text file is the right call for the same reasons it’s the right call for any accumulating time-series dataset:

  • Typed columns. Start and end times are timestamps; forecast text is a string. Parquet preserves those types across reads and writes, so there’s no parsing ambiguity.
  • Efficient compression. The detailedForecast field contains long strings with a lot of repeated meteorological vocabulary. zstd compression on columnar data compresses this extremely well — the archive for 480+ entries is a fraction of what equivalent JSON or CSV would cost.
  • Interoperable. The file can be loaded in pandas, queried directly with DuckDB, or read by any Parquet-aware tool — no Python-specific format (like pickle) required.
  • Deduplication is trivial. df.drop_duplicates() is one line. Doing the same thing cleanly against a growing JSON file or CSV requires reading the whole file and writing it back, with the same outcome but more code.

The alternative would be appending a row to a CSV on each run. That would work for simple queries, but you’d lose type information, pay for delimiter parsing on every read, and have no compression at all. For a dataset that grows by one row per day indefinitely, Parquet is the obvious choice.

The bigger picture

A scheduled GitHub Action that commits back to its own repository is a lightweight, zero-infrastructure alternative to a cron job on a server. The scheduler, the compute, and the storage are all provided by GitHub. The version history is git. There’s nothing to maintain.

The pattern scales to anything that produces small, structured output on a schedule: scraping a public API, generating a report, checking a service’s status, archiving a feed. As long as the output fits in a git repository and the job finishes in under the workflow timeout, a scheduled Action is all you need.

python-get-forecast has been running reliably since September 2022 — over 1,300 nightly runs, each completing in under 30 seconds, building up an archive of Pittsburgh weather history one forecast at a time.

Further reading