All Questions
Tagged with beautifulsoup selenium-webdriver
2,607
questions
0
votes
2
answers
84
views
scrape a website which has the same url for multiple pages? with the page jump being an ajax request
I've been at this for days, I'm trying to scrape this website: "https://careers.ispor.org/jobseeker/search/results/".
I've got everything covered, from the script that will extract the ...
0
votes
1
answer
38
views
selenium's driver gets wrong page in python
I am trying to scrape certain odds for a football tournament. To this end I wrote a piece of code which first generates the exact link I want and then loads the corresponding page. The problem is, the ...
- 111
-1
votes
0
answers
50
views
Using firefox selenium to scrape a page with infinite scroll resulting in error, possibly due to too much data
I'm trying to scrape this page with infinite scroll on meetup for a list of past events. I want to get a list of events including name, date, and URL (mostly just the name, the other 2 are optional).
...
- 53
-1
votes
1
answer
76
views
Issues with Web Scraping Automation in Python Using Selenium
I’m having trouble with my ETL process. Let me explain my problem, I have this code:
import time
from selenium import webdriver
from selenium.webdriver.common.by import By
import pandas as pd
import ...
-1
votes
0
answers
63
views
How to use BeautifulSoup and Selenium together correctly?
I'm trying to implement webscraping with tkinter, selenium, and beautifulsoup. My question is: Can I mix the two together? My approach is to use mostly beautifulsoup to log in, get elements, and ...
2
votes
2
answers
72
views
Trying to Web Scrape in Python
I am trying to extract a flow of the URL but am stuck. The URL from where I am extracting data, redirects to a page, which has the URL which then has to be clicked to go to the page's URL I actually ...
- 21
0
votes
2
answers
61
views
Problem with webscraping using selenium and beautifulsoup
I'm creating a price comparison website for my uni project. I'm trying to print the items and the price from this website https://www.lotuss.com.my/en/category/fresh-produce?sort=relevance:DESC but I ...
- 15
0
votes
2
answers
77
views
findAll() returning empty outputs
I'm trying to scrape the title, date, rating and actual review of each reviews form mouthshut.
But I'm unable to extract anything under the title of page.
The review is in tag under class 'more ...
- 13
0
votes
2
answers
80
views
Python-Scraper with BS4 and Selenium : Session-Issues with chrome
I am trying to grab the list of all the banks that are located here on this page
http://www.banken.de/inhalt/banken/finanzdienstleister-banken-nach-laendern-deutschland/1
note we've got 617 results
my ...
- 1,231
0
votes
2
answers
132
views
Why is my Selenium script only scraping data for 7 matches on Forebet?
I'm working on a web scraping project using Selenium to scrape football match data from a sports prediction website (let's use EXAMPLE for FOREBET). However, my script only retrieves data for 7 ...
-1
votes
1
answer
74
views
How I do scraping in a web page that show information with scroll and not with index?
I'm learning web scraping, and i'm trying to get data from a page that show information with scroll, What can I do in this scenario?, Is there a function to make the entire page load? I am using ...
1
vote
1
answer
63
views
How to get image src of carousel posts when scraping Instagram with Selenium
I'm trying to scrape Instagram photos with Selenium. The script is working to get the first image of all types of posts (single, video, carousels) but when I try to get the src of any subsequent ...
- 11
0
votes
2
answers
61
views
web scraper is not grabbing desired text
I am trying to scrape the sku and description on this site:
https://www.milwaukeetool.com/products/power-tools/drilling/drill-drivers
but, it wont scrape the desired elements despite the code being ...
0
votes
1
answer
56
views
Selenium WebDriverWait try/finally statement fails even if expected condition is met
I'm following the documentation on the Selenium website for how to wait for Ajax responses before proceeding, and while the correct dynamically loaded information is found, a timeout error is still ...
- 55
1
vote
1
answer
70
views
Error in extracting live tennis match information from the bwin.it website with Python, Selenium, and BeautifulSoup using selenium headless mode
I would like to extract the information of live tennis matches from this web page https://sports.bwin.it/it/sports/live/tennis-5 with Python, Selenium, and BeautifulSoup.
With this code it works well, ...
- 11