Data acquisition: how to collect good data from forms, files, APIs and the web

Every analysis is only as good as the data that goes into it. Data acquisition is the first step: collecting the right data, in the right format, legally and reliably.

Four common sources

Pulling data from an API with Python

import requests, pandas as pd

resp = requests.get("https://api.example.com/orders",
                    params={"from": "2026-10-01"},
                    headers={"Authorization": "Bearer YOUR_KEY"},
                    timeout=30)
resp.raise_for_status()
orders = pd.DataFrame(resp.json()["orders"])
orders.to_csv("data/orders_2026-10.csv", index=False)

Keep API keys out of your code (use environment variables) and never commit them to Git.

Build quality in at the start

• Validate at entry: drop-downs, required fields and email checks stop bad data early.
• Use consistent formats: dates as YYYY-MM-DD, one currency, one unit.
• Record where data came from and when, so results can be traced and refreshed.

Collect responsibly

If you collect personal data in India, the Digital Personal Data Protection (DPDP) Rules, 2025, notified in November 2025 with a phased 18-month compliance period, require clear, standalone notices stating the specific purpose, valid consent, and prompt notification of affected people after a data breach. Collect only what you need. For web pages, respect each site's terms of use and robots.txt, and prefer an official API or data export when one exists.

Next: once data is collected, it needs cleaning and moving into one place. That is data engineering.

Further reading

• Government notifies Digital Personal Data Protection Rules 2025 (Indian Infrastructure)
• Requests: HTTP for Humans (official docs)