Pandas how to get a datafrrame as JSON/dict format excluding NAs efficiently? [duplicate]

  dataframe, pandas, python

I would like a way to get a DataFrame in JSON/dict format excluding NAs efficiently.

This line of code is pretty fast, however when I want to exclude NAs from the results it makes the process very slow.

df.to_dict(orient="records")

I have tried different all the different answers here:

The fastest way to achieve this at the moment is this line of code:

[row.dropna().to_dict() for _, row in df.iterrows()]

However these solutions are very slow when dealing with millions of rows


Is there any way to get better time performance?

Source: Python Questions

LEAVE A COMMENT