How to extract text from a-html elements by using a customized function?

  function, html, python, web-scraping

I am trying to extract a text sub-elements for the first table element from an specific url.

The main goal is to iterate over all a sub-elements in order to extract #text as a list.
To accomplish this task I defined a function:

from bs4 import BeautifulSoup
import lxml
import requests

url = 'https://www.salario.com.br/profissao/abacaxicultor-cbo-612510/'

def getColumns(url):
    url = requests.get(url)
    soup=BeautifulSoup(url.text, 'lxml')
    table = soup.find('table', attrs={'class':'listas'})
    
      # finding table headers

    all_other_headers = table.find('thead').find('td').find_all('a')

    #cannot run this line as previous return a 'None' element#
    #colnames = [hdr.text for hdr in all_other_headers]#
   

    return all_other_headers

When applied getColumns(url) it returns an empty list :[], when I expect an output that looks like this: ['Sálario Mensal', 'Sálario Anual', 'Salário por Semana', 'Sálario por Hora']

Why if I am using a tag is still not working?
How could I possibly adjust this function?

Source: Python Questions

LEAVE A COMMENT