Listing Lifecycle

Learn how MarketCheck tracks vehicle listings from discovery to expiration, and understand the key date fields that govern a listing's status.

MarketCheck's platform captures and tracks the complete lifecycle of each vehicle listing, from its initial discovery on a dealer's website to its eventual expiration. This guide explains the daily process and the key date fields that manage a listing's status in our system.

The Lifecycle Process

Our daily crawl engine continuously monitors dealer websites to manage the lifecycle of each inventory listing. This process ensures that the data in our API and feeds accurately reflects the current vehicles available from dealers.

Listing Discovery

When our crawler finds a new vehicle on a dealer website, it creates a new listing in the MarketCheck database. This listing represents the vehicle's presence at that dealer and serves as the starting point for its lifecycle.

Daily Tracking

On subsequent daily crawls, if the same listing is found with unchanged price and mileage, we update its status_date to the current timestamp. The scraped_at and first_scraped_at dates, which mark discovery milestones, follow specific update rules. This daily "heartbeat" confirms the listing is still active.

Expiration and Change

The lifecycle ends when a listing is either removed from the dealer's website or its key attributes change.

Listing Removal: When a listing disappears from the site, we stop updating its status_date. Because our API and data feeds only expose listings updated within recent periods, the expired listing is automatically removed from active results.
Attribute Change: If a listing's price or mileage is modified by the dealer, MarketCheck creates a new listing record. The old listing expires, and a new listing begins its lifecycle.

Key Date Fields

You can track a listing's lifecycle using four distinct date fields that MarketCheck manages during the crawling process. Each field serves a specific purpose in tracking different aspects of a vehicle's presence in the market.

These fields are populated by our crawling system to track a listing's status and changes over time.

`first_scraped_at`

The timestamp when MarketCheck first discovered the VIN from any dealer
This value remains constant for a VIN throughout its entire history, even if the vehicle moves between dealers or is relisted years later
API alias: first_seen_at_mc

`scraped_at`

The timestamp when the listing was first created, either due to a new VIN discovery, attribute changes (price/mileage), or reappearing after being absent for more than 3 days
This date does not change throughout the listing's lifecycle
For a single VIN with multiple listings at a source, each listing will have a different scraped_at date
API alias: first_seen_at

`status_date`

The timestamp when the listing was last seen on the dealer's website with unchanged price, mileage, and other attributes
This date is updated daily to reflect continued presence and acts as a "heartbeat" for the listing
API alias: last_seen_at

`relisted_at_date`

The timestamp when the VIN was first listed on the dealer's website
On the first day of VIN discovery, this is the same as scraped_at.
As the listing is updated or reappears after absence, scraped_at will change to reflect the new listing date, but relisted_at_date remains unchanged
This field only resets when the VIN is completely removed from the dealer's website and reappears more than 30 days later.
API alias: first_seen_at_source

All date fields are provided in both Unix epoch seconds (*_at) and ISO-8601 format (*_at_date) in API responses.

Lifecycle Transitions

Understanding how listings transition between states helps you interpret the date fields and predict listing behavior.

New Listing Creation

A new listing is created when:

A VIN is discovered for the first time from any dealer (all four dates are identical)
Price or mileage changes on an existing listing (new scraped_at and status_date, while first_scraped_at and relisted_at_date remain from previous listing)
An existing VIN reappears after being absent for more than 3 days (new scraped_at and status_date, while first_scraped_at and relisted_at_date remain unchanged)

Daily Status Updates

For listings with no attribute changes:

status_date is updated daily to reflect continued presence
scraped_at, first_scraped_at, and relisted_at_date remain unchanged

Long-Term Absence and Reactivation

When a VIN reappears after extended absence:

Less than 30 days: Only scraped_at and status_date are updated; relisted_at_date remains unchanged
More than 30 days: relisted_at_date resets to match the new scraped_at date; first_scraped_at always remains the original discovery date

Example: If a VIN disappears on March 1st and reappears on April 5th (35+ days), the relisted_at_date will reset to April 5th, treating it as a fresh start on that dealer's website.

Special Scenarios

These edge cases demonstrate how the system handles complex situations that can occur in real-world dealer operations.

Dealer Transfers

When a vehicle moves to a new dealer, a new listing is created for the new dealer's website. The first_scraped_at value for that VIN remains unchanged, preserving the vehicle's initial discovery date across all dealer transfers. The relisted_at_date reflects when the VIN first appeared on the new dealer's website.

Data Conflicts

When multiple updates are received for the same VIN on the same day, the system uses the latest status_date timestamp to resolve conflicts and maintain data consistency.

Multi-Year Reactivation Example

Consider a VIN first discovered in 2020 that reappears after 5 years:

first_scraped_at remains the original 2020 discovery date
relisted_at_date, scraped_at, and status_date all reset to the 2025 reappearance date
If price changes occur later, first_scraped_at and relisted_at_date remain unchanged, while scraped_at and status_date update to reflect the new listing

All listing changes are preserved as separate records in our historical database, enabling complete audit trails and trend analysis for any VIN.

Example: Listing Lifecycle Demonstration

This example illustrates the various stages a vehicle listing goes through, highlighting the key date fields and their changes over time.

For brevity, only the relevant date fields are shown in each stage in human-readable format.

Day 1: 03-Feb-2019

A new car (VIN 1GHDT13W8V2711434) is seen for the first time. A new listing is created:

{
  "id": "1GHDT13W8V2711434-bab1-73540ae7798c",
  "vin": "1GHDT13W8V2711434",
  "first_scraped_at": "2019-02-03",
  "scraped_at": "2019-02-03",
  "status_date": "2019-02-03",
  "relisted_at_date": "2019-02-03",
  "price": 22000,
  "miles": 30000,
  "seller_name": "Langdale Ford Valdosta"
}

Day 2-3: 05 Feb 2019

The car is seen with the same price. Only status_date is updated:

{
  "id": "1GHDT13W8V2711434-bab1-73540ae7798c",
  "vin": "1GHDT13W8V2711434",
  "first_scraped_at": "2019-02-03",
  "scraped_at": "2019-02-03",
  "status_date": "2019-02-05",  <---- Only this is updated
  "relisted_at_date": "2019-02-03",
  "price": 22000,
  "miles": 30000,
  "seller_name": "Langdale Ford Valdosta"
}

Day 4: 06-Feb-2019

Price drops, a new listing is created:

{
  "id": "1GHDT13W8V2711434-sft7-87456aerrt36",  <---- New ID for new listing
  "vin": "1GHDT13W8V2711434",
  "first_scraped_at": "2019-02-03",
  "scraped_at": "2019-02-06",   <---- New scraped_at date
  "status_date": "2019-02-06",  <---- New status_date
  "relisted_at_date": "2019-02-03",
  "price": 20000,
  "miles": 30000,
  "seller_name": "Langdale Ford Valdosta"
}

The previous listing id 1GHDT13W8V2711434-bab1-73540ae7798c is now inactive, and stops getting updates, but it still exists in the database for historical reference.

Day 5-7: 09 Feb 2019

The car is seen with the same price. Only status_date is updated:

{
  "id": "1GHDT13W8V2711434-sft7-87456aerrt36",
  "vin": "1GHDT13W8V2711434",
  "first_scraped_at": "2019-02-03",
  "scraped_at": "2019-02-06",
  "status_date": "2019-02-09",    <---- Only this is updated
  "relisted_at_date": "2019-02-03",
  "price": 20000,
  "miles": 30000,
  "seller_name": "Langdale Ford Valdosta"
}

Day 21: 23-Feb-2019

Car had gone offline on day 8 and reappeared on day 21, maybe due to reconditioning, repairs, or other factors. A new listing is created:

{
  "id": "1GHDT13W8V2711434-b59d-70af266a9eda",  <---- New ID for new listing
  "vin": "1GHDT13W8V2711434",
  "first_scraped_at": "2019-02-03",
  "scraped_at": "2019-02-23",    <---- New scraped_at date
  "status_date": "2019-02-23",   <---- New status_date
  "relisted_at_date": "2019-02-03",
  "price": 20000,
  "miles": 30000,
  "seller_name": "Langdale Ford Valdosta"
}

Day 2363: 13-Aug-2025

Car has come back onto market after more than 6 years at different dealerships, a new listing is created:

{
  "id": "1GHDT13W8V2711434-re45-jh54add49ytr",  <---- New ID for new listing
  "vin": "1GHDT13W8V2711434",
  "first_scraped_at": "2019-02-03", <---- Original first_scraped_at date
  "scraped_at": "2025-08-13",    <---- New scraped_at date
  "status_date": "2025-08-13",   <---- New status_date
  "relisted_at_date": "2025-08-13",   <---- New relisted_at_date
  "price": 7000,
  "miles": 90000,
  "seller_name": "Group 5 Automotive"
}

Using Date Fields

Understanding how to use these date fields helps you make informed decisions when analyzing inventory data and tracking vehicle availability.

Use status_date (alias: last_seen_at) to determine when a vehicle was last active at a specific price and identify the most recent activity
Use scraped_at (alias: first_seen_at) to track when the current listing iteration began, whether from new discovery, attribute changes, or reappearance after absence
Use first_scraped_at (alias: first_seen_at_mc) to identify when a VIN first appeared in the MarketCheck system, regardless of dealer changes or gaps in availability
Use relisted_at_date (alias: first_seen_at_source) to understand when a VIN first appeared on a specific dealer's website, which resets only after 30+ day absences