Writing a (scraped) list of dataframes to single excel sheet

  excel, pandas, python, web-scraping

I am trying to scrape an HTML table off a list of webpages (there’s only one table on each page).

Here’s the first part of my code:

for i in range(len(urls)):
  r = requests.get(urls[i], header)
  data = pd.read_html(r.text,index_col=0, header=0) 
  appended_list.append(data)

‘urls’ is a list of urls (each of which have one static table)
This code works – I get a list of dataframes (appended_list).

Now, I want to write this list of dataframes into a single excel sheet. This is what I’ve tried –

writer = pd.ExcelWriter('test_1.xlsx', engine='xlsxwriter')   
row = 0

for df in appended_list:
  df.to_excel(writer,startrow=row,startcol=0)   
  row = row + len(df.index) + 1

writer.save()

I get the error:

    df.to_excel(writer,startrow=row,startcol=0)
AttributeError: 'list' object has no attribute 'to_excel'

What seems to be happening is that each element (dataframe) of the appended_list is still being read as a list. How do convert each of these list items to a dataframe so that they can be written on an excel file?

Thank you in advance.

Source: Python Questions

LEAVE A COMMENT