Skip to content

Download error fix - CensusCountyBusinessPatterns#1961

Open
kartik-s21 wants to merge 1 commit intodatacommonsorg:masterfrom
kartik-s21:download-error-fix
Open

Download error fix - CensusCountyBusinessPatterns#1961
kartik-s21 wants to merge 1 commit intodatacommonsorg:masterfrom
kartik-s21:download-error-fix

Conversation

@kartik-s21
Copy link
Copy Markdown

@kartik-s21 kartik-s21 commented Apr 17, 2026

The main.py script for CensusCountyBusinessPatterns was failing during the data download phase.The script calculates the default "data_end_year" as two years prior to the current year (in 2026, it defaults to 2026-2=2024), but the source has not yet published the 2024 dataset. This caused the script to crash with Error 404: File Not Found.

So what Is done -
Modified scripts/census_county_business_patterns/main.py to make the download process more resilient:

  • Wrapped a Try-Except Block over the dowload.
  • Instead of crashing, the script now logs a warning and continues to the next available file or year.

@kartik-s21 kartik-s21 requested a review from saanikaaa April 17, 2026 08:09
Copy link
Copy Markdown
Contributor

@gemini-code-assist gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request improves the reliability and security of data ingestion scripts by adding exception handling for network requests, implementing path traversal checks during zip extraction, and filtering for specific file extensions. It also includes minor code cleanups in the energy dataset scripts and test files. Feedback suggests addressing an indentation issue, removing an unused variable, and extending error handling to include corrupted zip files.

Comment thread scripts/census_county_business_patterns/main.py Outdated
Comment thread scripts/census_county_business_patterns/main.py Outdated
Copy link
Copy Markdown
Contributor

@saanikaaa saanikaaa left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see un_energy changes in this PR. Pls rectify and make separate PR for seperate imports

@saanikaaa saanikaaa self-requested a review April 21, 2026 06:28
@kartik-s21 kartik-s21 changed the title Download error fix Download error fix - CensusCountyBusinessPatterns Apr 21, 2026
@kartik-s21
Copy link
Copy Markdown
Author

I see un_energy changes in this PR. Pls rectify and make separate PR for seperate imports

Rectified to Two separate imports.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants