Web Scraping, Data Analytics & Data Intelligence Glossary

An Application Programming Interface (API) is a defined set of rules that lets one software system request data or services from another, often returning clean, structured data without needing to parse a webpage. Many companies offer APIs as a lawful, stable alternative to scraping their HTML directly.

Example: A retailer exposes a product API that returns JSON price and stock data instead of requiring scrapers to read the storefront page.

The practice of extracting data by calling a website's or app's underlying API endpoints instead of parsing rendered HTML, usually resulting in faster and more reliable extraction.

Example: Instead of scraping a shopping app's visible listings, a scraper calls the same backend API the app uses to load product data.

The process of removing or masking personally identifiable details from a dataset so individuals cannot be re-identified, while the remaining data stays useful for analysis.

Example: A dataset of customer reviews is anonymized by stripping names and email addresses before it's used for sentiment analysis.

Techniques websites use to detect and block automated data collection, such as CAPTCHAs, rate limits, IP blocking, and behavioral analysis.

Example: A travel site serves a CAPTCHA after detecting unusually fast, repetitive requests coming from a single IP address.

A name-value pair inside an HTML tag that provides extra information about an element, such as its class, id, or link target.

Example: A scraper targeting product prices extracts the value of a data-price attribute rather than the visible text on the page.

Information describing who a brand's customers or website visitors are, including demographics, interests, and online behavior, used to guide marketing and product decisions.

Example: An e-commerce brand combines its own audience data with scraped competitor traffic estimates to refine ad targeting.

Running data collection tasks on a schedule or trigger without manual intervention, typically through scripts, cron jobs, or dedicated scraping platforms.

Example: A price-monitoring tool automatically re-scrapes competitor sites every six hours and flags any price changes.

Combining data from multiple sources or records into a single summarized view, such as totals, averages, or counts.

Example: Sales figures scraped from ten regional marketplaces are aggregated into one weekly revenue report.

A visual interface that displays key metrics and trends in real time, usually through charts, tables, and KPIs, so users can monitor performance at a glance.

Example: A retail team watches a dashboard that updates hourly with scraped competitor pricing and stock levels.

A method of comparing two versions of a page, product, or campaign by showing each to a separate audience segment and measuring which one performs better.

Example: An online store A/B tests two product page layouts and adopts the version with the higher conversion rate.

Extremely large, fast-growing, or varied datasets that traditional tools struggle to process, typically defined by volume, velocity, and variety.

Example: Millions of scraped product listings updated daily across thousands of e-commerce sites qualify as big data.

An automated software program that performs repetitive tasks online, such as fetching pages, submitting forms, or extracting data, without human input.

Example: A pricing bot visits competitor websites every night to collect the latest listed prices.

Systems websites use to distinguish automated traffic from genuine human visitors, based on signals like request speed, mouse movement, and browser fingerprints.

Example: A site's bot detection flags a visitor that loads fifty pages in two seconds as non-human.

Deliberately slowing down data transfer speed, either by a website to limit scrapers or by a scraper to avoid overwhelming a target server.

Example: A responsible scraping job throttles its bandwidth so it doesn't slow the target site down for real visitors.

Running a data job on a large group of records at scheduled intervals rather than handling records one at a time in real time.

Example: A data pipeline batch-processes the previous day's scraped listings every night at 2 a.m.

The combination of tools, processes, and technology used to turn raw data into actionable insight for business decision-making.

Example: A BI platform blends internal sales data with scraped market data to show where a company is losing share.

A technique websites use to identify and track visitors, including bots, based on unique characteristics of their browser and device rather than cookies alone.

Example: A site blocks a scraper once its fingerprint, made up of screen size, fonts, and plugins, is flagged as repeated and non-human.

A challenge-response test used to determine whether a visitor is human, commonly image or text puzzles, designed to block automated bots.

Example: A scraper hits a CAPTCHA wall after too many rapid requests and needs a slower crawl rate to continue.

Temporarily stored data from a previous request, used to speed up future access and reduce repeated load on the source website.

Example: A scraping tool caches a page's HTML for an hour so it doesn't re-request unchanged content unnecessarily.

An unsupervised technique that groups similar data points together based on shared characteristics, without predefined labels.

Example: Clustering groups scraped customer reviews into themes like shipping complaints, pricing feedback, and product quality.

Running web scraping infrastructure on cloud servers instead of local machines, enabling greater scale, rotating IPs, and continuous uptime.

Example: A company runs its scraping jobs on cloud servers spread across regions to reliably collect region-specific pricing.

The practice of gathering and analyzing public information about competitors, such as pricing, product assortment, and marketing moves, to inform strategy.

Example: A brand tracks competitor pricing and promotions weekly through scraped data to adjust its own offers.

Running multiple scraping requests or processes at the same time to speed up data collection.

Example: Increasing concurrency from 5 to 50 simultaneous requests lets a crawler finish a 10,000-page job much faster.

A small piece of data a website stores in a visitor's browser to remember session information, login state, or preferences.

Example: A scraper reuses a saved login cookie so it doesn't need to sign in again on every request.

A browser security rule that restricts web pages from making requests to a different domain than the one that served the page, unless explicitly allowed.

Example: A frontend script calling a third-party API directly gets blocked by CORS, requiring a backend proxy instead.

An automated program that systematically browses and follows links across websites to discover and collect pages, often the first step before data extraction.

Example: A crawler starts at a marketplace's homepage and follows category links to discover every product page.

A scheduled task on a server that runs automatically at set times or intervals, commonly used to trigger recurring scraping jobs.

Example: A cron job kicks off a competitor price scrape every day at midnight.

A pattern used to target specific HTML elements on a page based on their tag, class, or id, commonly used to pinpoint data during scraping.

Example: A scraper uses the CSS selector .product-price to grab only the price element from each listing.

A plain text file format that stores tabular data with commas separating values, widely used for exporting and sharing scraped or analyzed datasets.

Example: Scraped product data is exported as a CSV file so it can be opened directly in Excel.

Tracking dark web sources for mentions of a company's data, brand, or credentials in order to detect leaks or threats early.

Example: A cybersecurity team monitors dark web forums for any mention of leaked customer records.

The process of examining raw data to find patterns, trends, and insights that support decision-making, using statistical and computational methods.

Example: A retailer runs data analytics on a year of scraped competitor prices to spot seasonal pricing patterns.

The process of fixing or removing incorrect, incomplete, duplicate, or inconsistent data before analysis.

Example: A dataset of scraped listings is cleaned to remove duplicate entries and fix inconsistent currency formats.

Adding new attributes or context to an existing dataset from external sources to make it more complete and useful.

Example: A company enriches its scraped lead list with firmographic data like company size and industry.

The process of pulling specific pieces of information out of a source, such as a webpage, document, or database, into a structured format.

Example: A scraper performs data extraction to pull the product name, price, and rating from each listing page.

The framework of policies, roles, and standards that ensure data is accurate, secure, and used properly across an organization.

Example: A data governance policy defines who can access scraped customer data and how long it can be stored.

A centralized storage repository that holds large volumes of raw data in its native format until it's needed for analysis.

Example: Raw scraped HTML and JSON files are dumped into a data lake before being cleaned and structured later.

The process of discovering patterns, correlations, and useful information within large datasets using statistical and machine learning techniques.

Example: Data mining on years of scraped job postings reveals which skills are rising fastest in demand.

An automated sequence of steps that moves data from a source through processing stages to its final destination, such as a database or dashboard.

Example: A data pipeline pulls scraped listings, cleans them, and loads them into a dashboard every morning.

The automated process of extracting data from websites, applications, or documents, typically by parsing HTML or calling underlying APIs.

Example: A market research firm uses data scraping to collect pricing across hundreds of online retailers daily.

Representing data graphically, through charts, graphs, or maps, to make patterns and trends easier to understand at a glance.

Example: A data visualization turns a year of scraped price data into a simple line chart showing seasonal trends.

A structured, centralized system for storing cleaned and organized data from multiple sources, optimized for reporting and analysis.

Example: Cleaned scraped data from dozens of sources is loaded into a data warehouse each night for BI reporting.

The process of identifying and removing duplicate records from a dataset to improve accuracy.

Example: Deduplication removes a product listing that appeared twice after being scraped from two different category pages.

A structured, tree-like representation of an HTML page that browsers and scraping tools use to locate and interact with specific elements.

Example: A scraper walks the DOM to find the div element containing a product's price.

A website whose content is generated or updated by JavaScript after the initial page load, requiring a headless browser to render before scraping.

Example: A dynamic website that loads prices via JavaScript needs a headless browser instead of a simple HTTP request.

Collecting product data such as prices, descriptions, stock levels, and reviews from online stores at scale.

Example: A brand runs e-commerce scraping across five marketplaces to track how its products are priced by resellers.

The process of converting data into a coded form that can only be read by someone with the correct decryption key, used to protect sensitive information.

Example: Scraped customer data is encrypted before storage to protect it from unauthorized access.

A specific URL where an API can be accessed to retrieve or send a particular type of data.

Example: A scraper calls a marketplace's product-detail endpoint directly instead of loading the full webpage.

A natural language processing technique that identifies and classifies key items in text, such as names, companies, locations, or dates.

Example: Entity extraction pulls company names and locations out of thousands of scraped news articles.

Code logic built into a scraper to detect and recover from failures like timeouts, blocked requests, or missing elements without crashing the entire job.

Example: Good error handling lets a scraper skip a broken page and continue collecting the rest of a 5,000-page job.

A data integration process that pulls data from a source, converts it into a usable format, and loads it into a target system like a warehouse.

Example: An ETL job extracts scraped listings, converts prices into a single currency, and loads the result into a warehouse.

An initial approach to analyzing datasets to summarize their main characteristics, often using visual methods, before formal modeling begins.

Example: Before building a pricing model, an analyst runs EDA on scraped price history to spot outliers and trends.

A continuous or scheduled stream of structured data delivered from a source to a destination system.

Example: A retailer subscribes to a scraped product feed that updates competitor prices every hour.

Narrowing a dataset down to only the records that meet specific criteria.

Example: A scraped dataset of job listings is filtered to show only postings from the last 30 days.

A pre-built software library that provides the core tools needed to build a scraper, such as request handling, parsing, and data storage.

Example: A developer builds a large crawler on top of a scraping framework instead of writing every component from scratch.

The visible, user-facing part of a website or application that renders in the browser, as opposed to the backend systems and APIs behind it.

Example: A scraper that only reads the frontend HTML may miss data that's loaded dynamically from the backend.

A technique for comparing text strings that aren't identical but are similar enough to be considered a match, useful for cleaning inconsistent data.

Example: Fuzzy matching links 'Coca-Cola Co.' and 'Coca Cola Company' as the same brand across scraped datasets.

A search technique that scans the complete text of documents or records to find matches, rather than only searching indexed fields.

Example: Full-text search lets an analyst find every scraped review mentioning 'late delivery' across a million records.

A European Union law governing how personal data of EU residents must be collected, processed, and protected.

Example: A company scraping publicly available data still reviews GDPR requirements before storing any personal details of EU users.

Information that identifies the physical location of a device, IP address, or user, often used to tailor content or analyze regional trends.

Example: Geolocation data from scraped listings shows which regions have the highest average product prices.

Configuring a scraper to access a website as if it were located in a specific country or region, often using local proxies, to see region-specific content or pricing.

Example: A brand uses geo-targeting to check if a competitor shows different prices to shoppers in the US versus the UK.

A widely used web analytics platform that tracks and reports website traffic, user behavior, and conversion metrics.

Example: A team compares its own Google Analytics traffic trends against scraped competitor traffic estimates.

A query language for APIs that lets a client request exactly the data fields it needs in a single call, rather than fetching a fixed structure.

Example: A scraper queries a site's GraphQL endpoint to pull only the product name and price fields it needs.

A web browser that runs without a visible user interface, controlled programmatically, commonly used to render JavaScript-heavy pages for scraping.

Example: A headless browser loads a dynamic pricing page fully, including its JavaScript content, before the scraper extracts the data.

A hidden link or field on a webpage, invisible to human visitors, placed specifically to detect and block automated bots that follow or fill it.

Example: A scraper that clicks a hidden honeypot link gets its IP address flagged and blocked.

The standard markup language used to structure content on web pages, made up of nested elements and tags that scrapers parse to extract data.

Example: A scraper reads the HTML of a product page to locate the tag containing the price.

A message sent from a client, such as a browser or scraper, to a server asking for a resource like a webpage or file.

Example: A scraper sends an HTTP request to a product page URL and receives the page's HTML in response.

A three-digit code returned by a server indicating the result of a request, such as 200 for success or 403 for forbidden access.

Example: A scraper repeatedly receiving a 429 status code knows it's being rate-limited and needs to slow down.

Organizing data with a structure that allows for faster search and retrieval, commonly used in databases and search engines.

Example: A database indexes scraped product IDs so lookups return results in milliseconds instead of seconds.

Combining data from different sources into a single, unified view for analysis or reporting.

Example: Data integration merges scraped competitor prices with internal sales data into one reporting system.

The practice of using data, tools, and analysis to build a clear, actionable understanding of markets, competitors, or operations.

Example: A company's data intelligence team turns scraped pricing, reviews, and job postings into a single competitor profile.

A unique numerical label assigned to a device connected to a network, used by websites to identify and sometimes block scraping traffic.

Example: A website blocks the IP address making thousands of requests per minute, suspecting a bot.

Cycling through multiple IP addresses during a scraping job so requests don't all appear to come from a single source, reducing the chance of being blocked.

Example: A large scraping job uses IP rotation across hundreds of proxies to avoid triggering rate limits.

Data generated by internet-connected devices and sensors, such as smart meters or wearables, often analyzed for patterns and automation.

Example: A logistics company combines IoT data from delivery trucks with scraped fuel price data to optimize routes.

The process of executing a page's JavaScript so that dynamically loaded content appears in the page's HTML, required before some pages can be scraped correctly.

Example: Without JavaScript rendering, a scraper may see an empty product list because the items load in after the initial page fetch.

A tool or system that automatically triggers scraping or data processing tasks at defined times or intervals.

Example: A job scheduler runs a full site crawl every Sunday night when traffic on the target site is lowest.

A lightweight, structured data format using key-value pairs, widely used by APIs and scrapers to store and exchange data.

Example: A scraped product's name, price, and rating are stored as a JSON object for easy processing.

A basic data structure where each unique key is linked to a specific value, commonly used in JSON, dictionaries, and databases.

Example: In scraped product data, "price": "49.99" is a key-value pair.

A text analysis technique that identifies the most relevant words or phrases within a document or dataset.

Example: Keyword extraction on scraped job postings reveals which skills employers mention most often.

A measurable value that shows how effectively a goal or objective is being achieved.

Example: A pricing team tracks 'average competitor price gap' as a KPI, built from daily scraped price data.

The delay between sending a request and receiving a response, which affects how quickly a scraper can collect data.

Example: High latency from an overseas proxy server slows down a scraping job that needs to run close to real time.

The process of identifying and collecting information about potential customers who may be interested in a product or service.

Example: A sales team uses scraped business directory data for lead generation, targeting companies matching their ideal customer profile.

Distributing requests across multiple servers or proxies to prevent any single one from being overwhelmed or flagged for unusual traffic.

Example: A scraping platform load-balances requests across dozens of proxy servers to keep each one under a safe request threshold.

Adapting data, content, or pricing analysis to reflect regional language, currency, or market conditions.

Example: A pricing report applies localization so scraped prices from five countries are shown in one common currency.

A record automatically generated by a system or scraper that tracks events, errors, and requests over time, useful for debugging.

Example: A scraper's log file shows exactly which URLs failed, and why, during last night's run.

A branch of artificial intelligence where systems learn patterns from data to make predictions or decisions without being explicitly programmed for every case.

Example: A machine learning model trained on scraped historical prices predicts likely price drops before a holiday sale.

Information about a market's size, trends, competitors, and customers, gathered and analyzed to guide business strategy.

Example: A market intelligence report combines scraped competitor pricing, reviews, and hiring trends into one strategic view.

Data that describes other data, such as a webpage's title tag, publish date, or file size, rather than the main content itself.

Example: A scraper collects a product page's metadata, like its last-updated date, alongside the visible price.

Software that sits between a scraper and the target website, or between components of a scraping system, to handle tasks like proxy rotation, retries, or header management.

Example: A proxy-rotation middleware automatically swaps IP addresses between each request in a scraping framework.

Running multiple threads of a program at once so a scraper can process several pages or tasks in parallel.

Example: Multithreading lets a scraper download ten pages simultaneously instead of one at a time.

Continuously or repeatedly tracking specific data points, such as prices or stock levels, over time to detect changes.

Example: A retailer's price monitoring system alerts the pricing team within minutes of a competitor dropping a price.

A field of AI focused on enabling computers to understand, interpret, and generate human language.

Example: NLP is used to automatically classify thousands of scraped customer reviews as positive, negative, or neutral.

A single point in the tree structure of an HTML document, representing an element, attribute, or piece of text that a scraper can target.

Example: A scraper selects the text node inside a span tag to extract a product's price.

The process of organizing or reformatting data into a consistent structure, such as standardizing units, formats, or scales.

Example: Normalization converts scraped prices listed in different currencies and formats into one standard format for comparison.

Extracting web data using visual, point-and-click tools rather than writing custom code, making scraping accessible to non-developers.

Example: A marketing analyst uses a no-code scraping tool to pull competitor prices without writing a single line of code.

Technology that converts images of text, such as scanned documents or screenshots, into machine-readable text data.

Example: OCR is used to extract prices from product images when a website blocks normal text-based scraping.

Data that is freely available for anyone to access, use, and share, often published by governments or organizations.

Example: A researcher combines open government data with scraped market data to study regional pricing trends.

Identifying data points that differ significantly from the rest of a dataset, which may indicate errors or genuinely unusual events.

Example: Outlier detection flags a scraped price that's 90% lower than usual as a likely data entry error on the source site.

The division of content across multiple pages, requiring a scraper to detect and follow 'next page' links to collect a complete dataset.

Example: A scraper handles pagination by automatically following the 'Next' link until it reaches the last page of results.

The process of analyzing raw HTML or text and converting it into a structured format that a program can work with.

Example: After fetching a page's HTML, a scraper parses it to pull out just the product title and price.

The actual data sent or received in a request, separate from headers or metadata.

Example: A scraper sends a search payload containing keywords and filters to a website's search API.

The use of historical data, statistics, and machine learning to forecast future outcomes or trends.

Example: Predictive analytics on two years of scraped pricing data forecasts when a competitor is likely to run its next sale.

An intermediary server that routes a scraper's requests through a different IP address, helping avoid detection, bypass geo-restrictions, or distribute load.

Example: A scraper routes its requests through a residential proxy server to appear as a genuine visitor from a specific city.

Information that is openly accessible without restriction, such as publicly listed prices, reviews, or business registrations.

Example: A price comparison tool only collects public data that any visitor to a website could see.

A popular programming language widely used to build web scrapers and data analysis tools due to its readability and extensive libraries.

Example: Many scraping frameworks, such as Scrapy and BeautifulSoup, are built in Python.

A request for specific data or information from a database, written in a structured query language.

Example: An analyst writes a query to pull every scraped listing priced under $50 from the database.

The part of a URL, after a question mark, that passes extra parameters such as search terms or filters to a website or API.

Example: A scraper builds a query string like ?category=shoes&page=2 to request a specific page of results directly.

The process of checking data or systems for accuracy, completeness, and consistency before they're used or delivered.

Example: A QA check compares a sample of scraped prices against the live website to confirm accuracy before a report ships.

A restriction that caps how many requests a client can make to a server within a set time period, used to prevent overload or abuse.

Example: A site's rate limiting allows only 60 requests per minute per IP address before returning an error.

Data that is collected, processed, and made available for use immediately or with minimal delay.

Example: A stock-tracking dashboard shows real-time data on inventory levels scraped every few minutes.

A sequence of characters that defines a search pattern, commonly used to find, match, or extract specific text within scraped content.

Example: A scraper uses a regex pattern to pull only the numeric price value out of a text string like 'Price: $49.99'.

A widely used API architecture style where data is accessed through standard HTTP methods like GET and POST, often returning data in JSON format.

Example: A scraper calls a marketplace's REST API to fetch product data directly rather than parsing the visible webpage.

A file websites publish to communicate which parts of the site automated crawlers are allowed or disallowed from accessing.

Example: Before crawling a site, a responsible scraper checks robots.txt to see which sections the site owner has restricted.

A program or tool that automatically collects data from websites, typically by sending requests and extracting information from the returned content.

Example: A custom-built scraper visits 500 product pages a day and saves the price and stock status of each.

A pattern, such as a CSS selector or XPath expression, used to pinpoint specific elements within an HTML document.

Example: A developer writes a selector targeting the h1 tag to extract each page's product title.

A technique that uses NLP to determine whether text expresses a positive, negative, or neutral opinion.

Example: Sentiment analysis on thousands of scraped product reviews shows customer opinion turned negative after a price increase.

A period of interaction between a client and a server during which state, like login status, is maintained, often tracked via cookies or tokens.

Example: A scraper maintains a logged-in session so it can access account-specific pricing pages without logging in on every request.

A file listing all the pages on a website, often in XML format, that helps crawlers discover content efficiently.

Example: A crawler reads a site's XML sitemap to quickly find every product page URL instead of following links manually.

Data organized in a clearly defined format, such as rows and columns or key-value pairs, that's easy to search and analyze.

Example: Once parsed, scraped product listings become structured data with clear fields for name, price, and rating.

The use of mathematical methods to collect, review, and interpret data in order to identify patterns and relationships.

Example: Statistical analysis of scraped pricing data confirms that weekend prices run about 8% higher than weekday prices on average.

Selecting a smaller, representative subset of a larger dataset for analysis, rather than processing the entire dataset.

Example: Instead of analyzing every scraped review, an analyst works with a random sample of 5,000 to estimate overall sentiment.

The defined structure of a dataset or database, specifying field names, data types, and relationships between them.

Example: A scraping project defines a schema with fields for product name, price, currency, and URL before collection begins.

The ability of a system, such as a scraping infrastructure, to handle increasing amounts of work, like more sites or higher request volume, without breaking down.

Example: A scraping platform built for scalability can grow from monitoring 10 websites to 10,000 without a redesign.

Data points collected or recorded at successive points in time, used to analyze trends and patterns over a period.

Example: Daily scraped prices for the same product form a time-series dataset that reveals seasonal pricing trends.

The process of breaking text into smaller units, such as words or phrases, for analysis, or in security, replacing sensitive data with non-sensitive placeholder tokens.

Example: Before running sentiment analysis, scraped review text is tokenized into individual words.

The study of visitor volume, sources, and behavior on a website to understand engagement and performance.

Example: Traffic analysis combines a company's own analytics with scraped estimates of competitor site visits.

Converting data from one format or structure into another to make it usable for analysis or storage.

Example: A transformation step converts scraped prices from multiple currencies into a single reporting currency.

Data that doesn't have a predefined format or organization, such as free-text reviews, images, or raw HTML.

Example: Scraped customer review text is unstructured data until it's processed and categorized.

The web address that specifies the location of a resource on the internet, used by scrapers to know exactly which page to request.

Example: A scraper reads a list of product URLs from a sitemap before visiting each one.

A string sent with a web request that identifies the browser, device, or bot making the request.

Example: A scraper sets its user agent to mimic a standard browser so it isn't immediately flagged as automated traffic.

The analysis of how users interact with a website or app, including clicks, scrolling, and navigation paths, to improve the user experience.

Example: UX analytics reveals that visitors often abandon a product page after seeing a price that looks high next to scraped competitor data.

Checking that data meets required rules and standards, such as correct format or expected ranges, before it's used.

Example: Data validation flags a scraped price of $0.00 as a likely extraction error rather than a genuine price.

Information about suppliers or third-party sellers, such as pricing, product range, and performance, used to evaluate and manage vendor relationships.

Example: A retailer compares vendor data scraped from multiple marketplaces before renegotiating supplier terms.

The sheer amount of data generated or collected, one of the core dimensions used to describe big data.

Example: A scraping operation covering thousands of sites daily generates a data volume too large for manual review.

The automated process of collecting data from websites by requesting pages and extracting specific information from their content.

Example: A market research company uses web scraping to track prices across hundreds of online retailers every day.

An automated message sent from one system to another when a specific event occurs, often used to trigger real-time updates.

Example: A price-monitoring tool sends a webhook to a Slack channel the moment a tracked competitor price changes.

Using technology to automatically carry out a series of connected tasks, such as scraping, cleaning, and reporting, with minimal manual input.

Example: Workflow automation moves scraped data from collection to a cleaned dashboard without anyone touching it manually.

A symbol used in a search or query to represent one or more unspecified characters, broadening the match.

Example: A wildcard search for 'phone*' returns scraped listings for phone cases, phone chargers, and phone stands.

A markup language that structures data using custom tags, commonly used for sitemaps, feeds, and data exchange between systems.

Example: A scraper reads a site's XML product feed instead of parsing individual HTML pages.

A query language used to navigate and select elements or attributes within an XML or HTML document.

Example: A scraper uses an XPath expression to select the third table cell in every row of a page's data table.

A human-readable data format often used for configuration files, including settings for scraping frameworks and pipelines.

Example: A scraping project's crawl settings, like request delay and target URLs, are defined in a YAML config file.

The percentage of attempted requests or pages that successfully return usable data during a scraping job.

Example: A scraper with a 98% yield rate successfully extracted data from 4,900 out of 5,000 attempted product pages.

Information a customer intentionally and proactively shares with a business, such as preferences given through a survey or quiz.

Example: A retailer combines zero-party data from a style quiz with scraped market trend data to personalize recommendations.

Location data tied to postal codes, often used to analyze regional pricing, demand, or demographic patterns.

Example: Zip code data layered onto scraped listings shows which regions consistently see higher average prices.