Category : python-requests

I am trying to open each company from StackOverflow companies and get specific informations(such as the whole description).Is there an easy way to do that using Beautiful Soup?For now I am getting the links of the companies of the first page. import requests from bs4 import BeautifulSoup r = requests.get(‘https://stackoverflow.com/jobs/companies’) src = r.content soup = ..

Read more

I am trying to scrape some data from daft.ie. My code: import requests import bs4 from requests import get from bs4 import BeautifulSoup import numpy as np import pandas as pd headers = { ‘User-Agent’: "Mozilla/5.0 (Windows NT 6.3; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/86.0.4240.198 Safari/537.36" } divs = np.linspace(start=0, stop=3120, num=157) divs = ..

Read more

import requests resp = requests.api.post(url,headers=self.headers,data=json.dumps(data)) when running it in jenkins, it is working fine sometimes, but in others it is throwing error requests.exceptions.ConnectionError: (‘Connection aborted.’, RemoteDisconnected(‘Remote end closed connection without response’)) Source: Python-3x..

Read more

Hello there i am trying to build a App Out of Digital Ocean API so basically i am sending a request to https://api.digitalocean.com/v2/droplets using the requests library(Python) and here’s my code import requests host = "https://api.digitalocean.com" headers = { "Content-Type": "application/json", "Authorization": "Bearer MYTOKEN" } dataa = {} api = requests.post(f"{host}/v2/droplets", headers= headers, data=json.dumps(dataa)) And ..

Read more

hi this is my code: >>> from bs4 import BeautifulSoup >>> import requests >>> url="http://localhost:8000/GetDetailsItem.html" >>> content=requests.get(url) >>> soup=BeautifulSoup(content.text) >>> print(soup) then i print it and it like as below: <!DOCTYPE html> <html> <head> <meta content="width=device-width" name="viewport"/> <title>سامانه ثبت اطلاعات املاک کشور</title> <link href="/Component/css/bootstrap.min.css" rel="stylesheet"/> as you can see above some parts is strange how ..

Read more

I’m working on my fortnite discord bot and I wanted to switch cosmetic API’s. But the API I wanted to switch to AKA https://fortniteapi.io isn’t letting me phrase the json with my code. Please someone help. Here is my code: async with aiohttp.ClientSession() as session: async with session.get(f"https://fortniteapi.io/items/list?name={item}", headers={‘Authorization’:’auth’}) as r: try: json = await ..

Read more

When using POST method to acquire data from tokopedia.com, I got the response.json() as: [{‘errors’: [{‘message’: ‘Request not allowed’, ‘extensions’: {}}]}] I tried saving this by replacing ‘https’ with ‘http’, which used to work in this case, and using JavaScript fetch to do the same jobs. Both failed to resolve this issue. This is the ..

Read more

I’m just doing some simple scrapping of Bing search results as practice and I was trying to insert custom cookies into the request to increase the page result sizes instead of scrapping through multiple pages. It may not be the most efficient but my motivation is just learning stuff at this point. def cookies(): _cookies ..

Read more