READING How to Scrape USAspending.gov Data Using Python (2026 Tutorial) 
On This Page

    How to Scrape USAspending.gov Data Using Python (2026 Tutorial) 

    How to Scrape USAspending.gov Data Using Python

    USAspending.gov is the U.S. government’s official public ledger for federal spending: every contract, grant, and loan awarded by every federal agency, searchable and free. For capture teams, market researchers, and GovCon platforms, it’s one of the richest sources of competitor and agency intelligence available anywhere, and unlike most government data sources, its API requires no authentication at all.

    That accessibility is also what makes people assume this is a simple scraping job. It isn’t, not once you need more than a handful of records, and not once you need the numbers to actually mean what you think they mean.

    In this tutorial, we’ll walk through exactly how to pull structured award data from USAspending’s API using Python, what the response looks like, and where the real difficulty shows up once you try to turn raw API output into something usable for competitor tracking or market research.

    What We’ll Pull From USAspending

    The core endpoint for award research is spending_by_award. Here’s what a structured pull typically captures for each award:

    FieldDescription
    Award IDUnique identifier for the award
    Recipient NameThe company or organization that won the award
    Award AmountTotal obligated award value
    Award TypeContract, grant, loan, or IDV vehicle type
    Awarding AgencyThe federal agency that made the award
    NAICS CodeIndustry classification for the awarded work
    PSC CodeProduct or service classification
    Period of Performance Start/End DateContract duration
    Place of PerformanceLocation where work is performed
    DescriptionFree-text description of the awarded work

    Don’t Want to Build and Maintain This Yourself?

    You can follow the steps below and run this manually, or you can skip straight to clean, structured data.

    At ScrapeInsight, we extract and normalize federal award data from USAspending at scale, matched to the agencies, NAICS codes, and competitors you actually care about.

    If you need ongoing, deduplicated award data instead of a script you have to babysit, talk to us today.

    Award Type Codes: The Detail That Changes What Your Data Means

    Before writing any code, it’s worth understanding award type codes, because filtering on the wrong ones will quietly return the wrong kind of data without any error telling you so.

    CodeAward TypeWhat It Means
    A, B, C, DContracts (definitive contracts, purchase orders, delivery orders, BPA calls)Direct procurement awards to a vendor
    IDV-prefixedContract vehiclesThe underlying vehicle, not the individual order against it
    02Block grantMoney usually goes to a state or territory first, not directly to a competitive applicant
    03Formula grantAllocated by formula, not won through competition
    04Project grantTypically the closest match for competitive, direct-to-recipient awards
    05Cooperative agreementCompetitive assistance with heavier federal involvement than a standard grant

    How to Find and Use USAspending’s API Endpoint

    Unlike most e-commerce or government sites, USAspending doesn’t require any reverse engineering to find its API. The endpoints are publicly documented at api.usaspending.gov, and no API key or authentication header is needed to query them.

    The endpoint that answers most practical questions, “who won what, from which agency, in which NAICS code, for how much”, is:

    https://api.usaspending.gov/api/v2/search/spending_by_award/
    

    It accepts a POST request with a JSON body specifying filters, the fields you want returned, sort order, and pagination.

    How to Scrap USAspending Award Data Using Python

    Step 1: Set Up Your Python Script

    import requests
    import time
    import json
    
    API_URL = "https://api.usaspending.gov/api/v2/search/spending_by_award/"
    naics_code = "541512"          # Your target NAICS code
    agency_name = "Department of Veterans Affairs"
    pages = 5                       # Number of pages to pull
    limit = 100                     # Records per page
    delay = 1.0                     # Delay between requests
    output_file = "usaspending_awards.json"
    

    Step 2: Build the Request Payload

    The API composes filters together, so you can narrow by time period, award type, NAICS code, agency, and more in a single request.

    def build_payload(page, award_type_codes):
        return {
            "filters": {
                "time_period": [
                    {"start_date": "2024-01-01", "end_date": "2026-01-01"}
                ],
                "award_type_codes": award_type_codes,  # contract codes, or ["04","05"] for competitive grants
                "naics_codes": [naics_code],
                "agencies": [
                    {"type": "awarding", "tier": "toptier", "name": agency_name}
                ]
            },
            "fields": [
                "Award ID",
                "Recipient Name",
                "Award Amount",
                "Awarding Agency",
                "Start Date",
                "End Date",
                "NAICS",
                "Description"
            ],
            "sort": "Award Amount",
            "order": "desc",
            "page": page,
            "limit": limit
        }
    


    Pass [“A”, “B”, “C”, “D”] for contract research, or [“04”, “05”] if you’re specifically researching competitive grant awards rather than procurement contracts.

    Step 3: Loop Through Pages and Collect Results

    headers = {"Content-Type": "application/json"}
    all_awards = []
    
    for page in range(1, pages + 1):
        print(f"[INFO] Fetching page {page}...")
        payload = build_payload(page, award_type_codes=["A", "B", "C", "D"])
    
        try:
            response = requests.post(API_URL, headers=headers, json=payload)
            print(f"[DEBUG] Status Code: {response.status_code}")
    
            if response.status_code != 200:
                print(f"[ERROR] Page {page} failed with status {response.status_code}")
                break
    
            data = response.json()
            results = data.get("results", [])
    
            if not results:
                print(f"[INFO] No more results at page {page}")
                break
    
            all_awards.extend(results)
            time.sleep(delay)
    
        except Exception as e:
            print(f"[ERROR] Exception on page {page}: {e}")
            break
    
    print(f"\n{len(all_awards)} awards collected.")
    

    Step 4: Save the Data 

    with open(output_file, "w", encoding="utf-8") as f:
        json.dump(all_awards, f, indent=2, ensure_ascii=False)
    
    print(f"Saved to {output_file}")
    

    What a USAspending Award Record Looks Like 

    {
      "Award ID": "CONT_AWD_36C24924D0021_3600_-NONE-_-NONE-",
      "Recipient Name": "ACME HEALTH SYSTEMS LLC",
      "Award Amount": 4820000.00,
      "Awarding Agency": "Department of Veterans Affairs",
      "Start Date": "2024-03-15",
      "End Date": "2026-03-14",
      "NAICS": "541512",
      "Description": "IT SYSTEMS INTEGRATION AND SUPPORT SERVICES"
    }
    

    Where This Breaks Down Once You Need It for Real Work

    The script above works. It will pull real award records, and for a one-off lookup or a proof of concept, that’s genuinely enough. The problems show up the moment this needs to power an actual capture process, competitor tracking system, or market research product.

    • Award amounts aren’t the same as money spent: The field returns the total obligation, not what’s actually been disbursed. A $5 million award can show the full amount obligated while only a fraction has been drawn down.
    • Modifications inflate your counts: A single award generates multiple transaction records as it gets modified or extended. Pulling raw records without separating base awards from modifications is a well-documented way to overcount how many distinct awards actually exist.
    • Names don’t normalize on their own: The same company or agency shows up under multiple name variants across records (“Department of Veterans Affairs” vs. “VA,” for example). Without a normalization layer, one competitor looks like several.
    • Data quality has real, documented gaps: A federal watchdog review found a meaningful share of obligations reported with incomplete program classification, and some agencies failing to submit required linkage data. Older and smaller awards are especially prone to missing fields.
    • There’s a real reporting lag: New awards typically take 30 to 90 days to appear after they’re issued, so this isn’t a live feed no matter how the pipeline is built.
    • Subaward data undercounts activity: Reporting compliance on subawards is uneven across prime recipients, on top of the shorter historical archive window.
    • Bulk historical pulls need a different endpoint entirely: Multi-year exports go through an asynchronous bulk_download job, not the interactive search endpoint, with its own polling and failure handling.
    • Joining with SAM.gov isn’t a clean match: USAspending covers what’s been awarded; SAM.gov covers what’s currently open. Matching them means reconciling different identifiers, and SAM.gov’s award API requires a key that can take one to two weeks to issue. Its notices also link out to the actual RFP and SOW documents, which a pipeline needs to explicitly capture rather than skip.
    • None of this is solved once: Naming conventions and data gaps don’t clean themselves up over time. A script that worked in January needs monitoring in July to confirm it’s still returning complete results.
    Related Read

    Looking for Government Tender Data Beyond USAspending.gov?
    USAspending.gov provides federal spending and award information. To explore public procurement opportunities beyond awarded contracts, learn how to collect government tender data from different procurement sources.

    How to Scrape Government Tender Contracts: Get Public Procurement Data

    Want Clean, Structured USAspending Data Without Maintaining a Pipeline?

    This script is a solid starting point for a single research question or a proof of concept. Keeping it reliable for ongoing competitor tracking, agency spend analysis, or a live GovCon product is a different commitment: award type interpretation, recipient and agency name normalization, modification deduplication, subaward and bulk download handling, and matching against SAM.gov’s open opportunities all add real engineering overhead most teams don’t want to own.

    At ScrapeInsight, we build managed USAspending data pipelines matched to the agencies, NAICS codes, and competitors you actually track, normalized and delivered on a schedule that fits how your team works, whether that’s a CRM feed, a research database, or a live product.

    Talk to us today and we’ll build you a data sample around your specific research needs.

    Keval Kothari

    Keval Kothari

    Founder and CEO