Listing Lifecycle

Overview

The listing lifecycle is the complete process an inventory record undergoes in the MarketCheck database, from its initial creation to its eventual expiration and archival. This lifecycle ensures that inventory data remains accurate, timely, and actionable for all downstream consumers.

• Creation: A new listing is ingested from a dealer's website or data feed. The system assigns a unique id, and captures initial attributes such as price, miles, seller_name, and key dates (status_date, scraped_at, first_scraped_at).
• Active: The listing is available for search, filtering, and display. It is considered current and visible to users and API clients.
• Update: Any change in price, miles, or other tracked attributes triggers a new record. The previous record is retained for history, and the new record reflects the updated values and dates.
• Expiration: When a listing is no longer found on the dealer's site for a defined period, it is marked as expired. Expired listings are not shown in active inventory but are preserved for historical and analytical purposes.

Detailed Field Definitions

• status_date: Updated when price or miles change, or when a listing reappears after a gap.
• scraped_at: Updated each time the listing is seen with the same price/miles.

Lifecycle Transitions and Triggers

• New Listing: Created when a dp_url is seen for the first time from any dealer.
• Attribute Update: If price or miles change, a new record is created with updated status_date and scraped_at.
• No Change: If the listing is seen again with no changes, only status_date is updated to reflect continued presence.
• Temporary Disappearance: If a listing disappears for up to 3 days and reappears with no changes, status_date is updated. If it reappears after more than 3 days, a new record is created.
• Expiration: If a listing is not seen for a defined period (e.g., 30 days), it is marked as expired and removed from active inventory.

Example: Listing Lifecycle in Action

The following scenario demonstrates how these dates are updated in the database based on price changes. The same logic applies when miles change.

Day 1: 03-Dec-2024

A new vehicle is seen for the first time. A new record is created:

{
  "id": "6accbdb05bfcedad821b75a3a5edf94d-4bae0aac-6f12",
  "dp_url": "https://www.meyertruckcenter.com/trucks/index.cgi/listing?ID=1143",
  "first_scraped_at": "2024-12-03",
  "scraped_at": "2024-12-03",
  "status_date": "2024-12-03",
  "price": 170000,
  "miles": 14000,
  "seller_name": "Meyer Truck Center Inc"
}

Day 2-3: 04-05 Dec 2024

Vehicle is seen with the same price. Only status_date is updated:

{
  "id": "6accbdb05bfcedad821b75a3a5edf94d-4bae0aac-6f12",
  "dp_url": "https://www.meyertruckcenter.com/trucks/index.cgi/listing?ID=1143",
  "scraped_at": "2024-12-03",
  "status_date": "2024-12-05",
  "price": 170000,
  "miles": 14000,
  "seller_name": "Meyer Truck Center Inc"
}

Day 4: 06-Dec-2024

Price changes. A new record is created:

{
  "id": "4bdb029bb4e6dad44a4827c7c24e1166-f3f277a4-56c0",
  "dp_url": "https://www.meyertruckcenter.com/trucks/index.cgi/listing?ID=1143",
  "scraped_at": "2024-12-06",
  "status_date": "2024-12-06",
  "price": 175000,
  "miles": 14000,
  "seller_name": "Meyer Truck Center Inc"
}

Day 5-7: 07-09 Dec 2024

Car is seen with the same price. Only status_date is updated:

{
  "id": "4bdb029bb4e6dad44a4827c7c24e1166-f3f277a4-56c0",
  "dp_url": "https://www.meyertruckcenter.com/trucks/index.cgi/listing?ID=1143",
  "scraped_at": "2024-12-06",
  "status_date": "2024-12-07",
  "price": 175000,
  "miles": 14000,
  "seller_name": "Meyer Truck Center Inc"
}

Day 8-9: 10-11 Dec 2024

Car disappears and reappears within 3 days, no price change. status_date is updated:

{
  "id": "4bdb029bb4e6dad44a4827c7c24e1166-f3f277a4-56c0",
  "dp_url": "https://www.meyertruckcenter.com/trucks/index.cgi/listing?ID=1143",
  "first_scraped_at": "2024-12-03",
  "scraped_at": "2024-12-06",
  "status_date": "2024-12-11",
  "price": 175000,
  "miles": 14000,
  "seller_name": "Meyer Truck Center Inc"
}

Day 21: 23-Dec-2024

Car reappears after more than 3 days. A new record is created:

{
  "id": "b696da902adc6d937de4679067387d4b-0e5781ca-0a3b",
  "dp_url": "https://www.meyertruckcenter.com/trucks/index.cgi/listing?ID=1143",
  "first_scraped_at": "2024-12-03",
  "scraped_at": "2024-12-23",
  "status_date": "2024-12-23",
  "price": 178000,
  "miles": 310,
  "seller_name": "Meyer Truck Center Inc"
}

Usage of Date Fields

• Use status_date to determine when a vehicle was available at a specific price.
• Use scraped_at to track the last time a listing was seen with unchanged price/miles.