I am trying to scrape an HTML table off a list of webpages (there’s only one table on each page).
Here’s the first part of my code:
for i in range(len(urls)): r = requests.get(urls[i], header) data = pd.read_html(r.text,index_col=0, header=0) appended_list.append(data)
‘urls’ is a list of urls (each of which have one static table)
This code works – I get a list of dataframes (appended_list).
Now, I want to write this list of dataframes into a single excel sheet. This is what I’ve tried –
writer = pd.ExcelWriter('test_1.xlsx', engine='xlsxwriter') row = 0 for df in appended_list: df.to_excel(writer,startrow=row,startcol=0) row = row + len(df.index) + 1 writer.save()
I get the error:
df.to_excel(writer,startrow=row,startcol=0) AttributeError: 'list' object has no attribute 'to_excel'
What seems to be happening is that each element (dataframe) of the appended_list is still being read as a list. How do convert each of these list items to a dataframe so that they can be written on an excel file?
Thank you in advance.
Source: Python Questions