Empowering developers to scrape the web at large-scale by utilizing a massive cache of public web pages
# initialize nokogiri
nokogiri = Nokogiri.HTML(content)
# get the listings
listings = nokogiri.css('ul.b-list__items_nofooter li.s-item')
# loop through the listings
listings.each do |listing|
# save the product info to outputs.
outputs << {
_collection: "products",
title: listing.at_css('h3.s-item__title')&.text,
price: listing.at_css('.s-item__price')&.text
}
# enqueue more pages to be scraped
pages << {
url: item_link['href'] unless item_link.nil?,
page_type: 'details'
}
end
Empowering decision makers to discover and understand knowledge through instant answers to their questions
Say hi and tell us more about your needs
© 2018 AnswersEngine.com