Hats are one of the most popular item cateogires flipped on Ebay. Price Finder is an advanced computer vision application that leverages the YoloV8 object detection model, webscraping, and Ebay's developer API to accurately determine the potential resale value of individual hats within bulk hat lots found on Ebay. By utilizing Price Finder, ebay sellers can automate part of their sourcing workflow and increase their daily listing and sale rate.
To use Price Finder you need to have an approved eBay developer account. This step won't be neccessary in the future when the backend of the project is hosted.
- Navigate to https://developer.ebay.com/develop and follow the instructions for setting up an account.
- After creating your account follow these steps to create an API keyset
- Navigate to Application keys and copy down your Client ID and Client Secret
- Navigate to User Access Tokens create a user token, and copy down your OAuth User Token
Chrome webdriver is needed inorder for the project to webscrape Ebay's website and aquire images of hat lots to analyze. After downloading the webdriver from the link above, unzip the file and copy down the absolute path to the webdriver. For example, "/Users/varunwadhwa/Downloads/chromedriver-mac-arm64/chromedriver"
-
Clone the repo
git clone https://github.com/varunUCDavis/Price-Finder.git
-
Modify the following fields in
config.yamlwith your API and system path informationchrome_driver_path: 'ENTER YOUR WEBDRIVER PATH' path: 'ENTER THE ABSOLUTE PATH TO YOUR PROJECT FOLDER' client_id: "ENTER YOUR CLIENT ID" client_sercret: "ENTER YOUR CLIENT SECRET" oauth_token: "ENTER YOUR OAUTH USER TOKEN"
-
Run the main.py script
python main.py
The model being used to detect individual hats within the bulk lot images has already been trained. However, if you would like to add additional training data and retrain the model, follow these steps.
- Add your training imgages and labels to "train/images" and "train/labels" respectively
- Run the train.py script
python train.py
Inorder to obtain images of bulk lot hats to analyze for potential resale value, Selenium and Chrome Driver were used to search for hat lots on ebays website, and scrape the results page.
def getLots(cls) -> dict:
items = {}
cls.driver.get(cls.EBAYLINK)
# Get all listing elements
# Wait for the page to load and the elements to be present
wait = WebDriverWait(cls.driver, 10)
wait.until(EC.presence_of_element_located((By.ID, 'srp-river-results')))
listings = cls.driver.find_elements(By.CSS_SELECTOR, "#srp-river-results .s-item__link")
listing_hrefs = [listing.get_attribute('href') for listing in listings]
# Loop through each listing link
for idx, listing_href in enumerate(listing_hrefs[:cls.NUMIMAGES]): # Limiting to the first 10 listings for this example
# Open the listing page
cls.driver.get(listing_href)
# Now on the listing page, wait for the main image to load and then scrape it
main_image = cls.driver.find_element(By.CSS_SELECTOR, "div.ux-image-carousel-item.image-treatment.active.image img")
src = main_image.get_attribute('src')
# Download the image content
response = requests.get(src)
if response.status_code == 200:
# Load the image content into an object that PIL can open
image = Image.open(BytesIO(response.content)).convert('RGB')
img_name = f"Lot{idx+1}"
items[img_name] = [image, listing_href]
# Go back to the search results page before proceeding to the next listing
cls.driver.back()
# Close the WebDriver
cls.driver.quit()
return itemsInorder to detect individual hats from a bulk lot image, the yolov8 object detection model was augemnted with labeled images of hats. If the algorithm detects hats with a confidece level of above 65%, a bounding box is drawn with the predicted coordinates.
After obtaining the images of individual hats from bulk lot imgaes, Ebay's image search API was used to find live listings of similar hats to each invidual hat. The first ten results of each search were used to estimate the potential selling price of the hat. The estimate was calculated by taking the first ten results' listing price, eliminating the outliers (outisde 1.5 IQR), and average the remaining results.
class PriceFinder:
with open("config.yaml", 'r') as file:
config = yaml.safe_load(file)
# Replace these with your actual eBay API credentials
CLIENT_ID = config['client_id']
CLIENT_SECRET = config['client_secret']
OAUTH_TOKEN = config['oauth_token']
# eBay Browse API endpoint for image search
API_ENDPOINT = 'https://api.ebay.com/buy/browse/v1/item_summary/search_by_image?&limit=10&conditions=USED'
# Path to your image file
IMAGE_PATH = '/Users/varunwadhwa/Desktop/ebayScrapper2/image4.png'
# Prepare the headers
headers = {
'Authorization': f'Bearer {OAUTH_TOKEN}',
'X-EBAY-C-MARKETPLACE-ID': 'EBAY_US',
'X-EBAY-C-ENDUSERCTX': 'affiliateCampaignId=<ePNCampaignId>,affiliateReferenceId=<referenceId>',
'Content-Type': 'application/json'
}
filters = 'sold:true,conditions:USED'
@classmethod
def mean_without_outliers(cls, data):
import numpy as np
# Calculate Q1 (25th percentile) and Q3 (75th percentile)
Q1 = np.percentile(data, 25)
Q3 = np.percentile(data, 75)
# Calculate the Interquartile Range (IQR)
IQR = Q3 - Q1
# Define outliers as those outside of Q1 - 1.5 * IQR and Q3 + 1.5 * IQR
lower_bound = Q1 - 1.5 * IQR
upper_bound = Q3 + 1.5 * IQR
# Filter out outliers and calculate the mean of the remaining data
filtered_data = [x for x in data if lower_bound <= x <= upper_bound]
# Return the mean of the filtered data
if filtered_data: # Check if the list is not empty
return np.mean(filtered_data)
else:
return None # Return None if all data are outliers or list is empty
@classmethod
def find_prices(cls, img):
prices = []
# Prepare the payload with the base64-encoded image
payload = {
'image': base64.b64encode(img).decode('utf-8')
}
# Make the POST request to the eBay API
response = requests.post(cls.API_ENDPOINT, headers=cls.headers, json=payload)
# Check if the request was successful
if response.status_code == 200:
# Parse the response data
items = response.json()
# Check if any items found
if 'itemSummaries' in items and len(items['itemSummaries']) > 0:
for item in items['itemSummaries']:
prices.append(float(item['price']['value']))
return cls.mean_without_outliers(prices)
else:
print("No items found.")
else:
print("Failed to search by image. Status code:", response.status_code, "Response:", response.text)