All Questions
Tagged with beautifulsoup screen-scraping
480
questions
0
votes
2
answers
54
views
Can't scrape links from the dynamic website using Selenium
I'm trying to collect links to personal profiles and contacts from the following website:
https://www.dlapiper.com/en-us/people#t=All&sort=relevancy&numberOfResults=100&f:CountriesID=[...
0
votes
1
answer
57
views
Extract the review ratings from i-herb webpage using python
I am facing a problem in trying to extract the ratings in the following webpage.
https://gr.iherb.com/r/panoxyl-acne-foaming-wash-10-benzoyl-peroxide-maximum-strength-5-5-oz-156-g/103844
When I try to ...
-1
votes
1
answer
55
views
how to extract a specific text and save in dataframe
I am a newbie in web scraping. I have managed to make this code, but there is no way to get the text
Any help or advice?
import requests
from bs4 import BeautifulSoup
import pandas as pd
url = 'https:...
- 1
1
vote
2
answers
43
views
problem in web scabing using BeautifulSoup
my problem is i can't catch the divs of class listing_LinkedListingCard__5SRvZ i use find_all however the result is empty list
the link :https://sa.aqar.fm/%D9%81%D9%84%D9%84-%D9%84%D9%84%D8%A8%D9%8A%...
- 57
0
votes
2
answers
56
views
Getting Links from a Dynamic Webpage
I am trying to get all links from a JS-based dynamic webpage with a load more option. I think I trigger THE page to scroll down. But, I could not get all the links. It turns only 103 links. I need to ...
1
vote
1
answer
42
views
Issue scraping dynamic table data from Human Microbiom Project (HMP) using python beautiful soup and selenium
I am working to web scrape the dynamic table data from the 'File UUID' column from the HMP website using python (beautiful soup and selenium). For some reason, I am able to pull all the data from the ...
-1
votes
1
answer
34
views
telguarder.com/ scraping with Python
I am trying to scrape telguarder.com for DID status result located in specific CSS selector. And, exactly that selector returns such error. I have tried other selector from the same web page and it ...
- 1
1
vote
0
answers
101
views
Data scraping and I get this problem: We're sorry but viewer-app doesn't work properly without JavaScript enabled. Please enable it to continue
I am attempting to scrape https://coworking.routesgrow.com/ .
When attempting to scrape with beautiful soup and normal requests library it wouldn't work. I switched to using selenium and beautiful ...
- 31
0
votes
1
answer
264
views
Unable to retrieve new html code after clicking next button using selenium. URL doesn't change as well
I am scraping https://www.coworker.com/search/turkey/izmir using selenium and beautiful soup. The html is rendered using Javascript which is why I am also using selenium. When clicking on the next ...
- 31
0
votes
1
answer
37
views
scraping nested xml using beautiful soup
xml = """<f transform="translate(7,7)" class="SoccerPlayer SoccerPlayer-11 Team-Away Outcome-Complete" data-id="8">
<rect x="-15" y=...
- 61
1
vote
1
answer
33
views
I am trying to screen scrape the Date and GDPnow number from the Atlanta Fed
https://www.atlantafed.org/cqer/research/gdpnow
I am trying to screen scrape the current GDPnow number with the date from the Atlanta Fed. Currently "Latest estimate: 3.5 percent — January 20, ...
- 11
1
vote
2
answers
121
views
Data/Table Scraping from Website using Python
I'm trying to scrape a data from a table on a website.
However, I am continuously running into "ValueError: cannot set a row with mismatched columns".
The set-up is:
url = 'https://kr....
0
votes
1
answer
45
views
Fetching Text in NavigableString in BeautifulSoup (bs4)
I am trying to parse an rss source and for that I need to get an image url.
One problem I run into is, sometimes image urls are not directly supplied with a tag but has been put inside of description ...
2
votes
1
answer
89
views
couldn't download table dataset from https://www.nordpoolgroup.com/en/Market-data1/#/nordic/table
I am working on my data science homework. I am trying to download the weekly table data by using pyython from : https://www.nordpoolgroup.com/en/Market-data1/Dayahead/Area-Prices/ALL1/Hourly12/?view=...
- 21
0
votes
1
answer
160
views
How to access the custom div's inside the div class?
Attached is a snip of the page which I am trying to access using BeautifulSoup and Python.
I want to the list of data-clipid and data-cliphref. But not able to move beyond with the code showed below.
...
- 163