Collectives™ on Stack Overflow

Find centralized, trusted content and collaborate around the technologies you use most.

Teams

Q&A for work

Connect and share knowledge within a single location that is structured and easy to search.

I have created a list datatype which has the path of three folders where each folder has a lot of .txt files. I am trying to work with each file in the folder by making it a pandas dataframe but I am getting the error as listed.

CODE-

for l in list: 
    for root, dirs, files in os.walk(l, topdown=False):
        for name in files:
            #print(os.path.join(root, name))
            df = pd.read_csv(os.path.join(root, name))   
ERROR- 
Traceback (most recent call last):
      File "feature_drebin.py", line 18, in <module>
        df = pd.read_csv(os.path.join(root, name))
      File "E:\anaconda\lib\site-packages\pandas\io\parsers.py", line 709, in parser_f
        return _read(filepath_or_buffer, kwds)
      File "E:\anaconda\lib\site-packages\pandas\io\parsers.py", line 449, in _read
        parser = TextFileReader(filepath_or_buffer, **kwds)
      File "E:\anaconda\lib\site-packages\pandas\io\parsers.py", line 818, in __init__
        self._make_engine(self.engine)
      File "E:\anaconda\lib\site-packages\pandas\io\parsers.py", line 1049, in _make_engine
        self._engine = CParserWrapper(self.f, **self.options)
      File "E:\anaconda\lib\site-packages\pandas\io\parsers.py", line 1695, in __init__
        self._reader = parsers.TextReader(src, **kwds)
      File "pandas/_libs/parsers.pyx", line 565, in pandas._libs.parsers.TextReader.__cinit__
    pandas.errors.EmptyDataError: No columns to parse from file
.txt file
                Can you show which file is causing this? May be you need delim_whitespace=True or sep = " " and header=None. Hard to tell without the content of the file.
– harvpan
                May 14, 2018 at 14:45
                If you just have a bunch of words on each line, pd.read_csv("filename.txt") would not show you error(s).
– harvpan
                May 14, 2018 at 14:55
I had the same problem and the answer was above:
"This error will also come if you are reading the csv which you have just created"
I have a rubbishy csv file created elsewhere where I have no control.
The file starts with two meaningless (at least useless to me) lines, two blank lines, then the data with column headings of phrases rather than words.  i.e. column headings each with multiple words with spaces. To anyone with a data background, that is a big NO. If you have column headings with spaces in them you are asking for problems; always use single words.
My plan for this csv was to open it, delete the first five rows and write the remaining lines to a newly created csv to which I had already written the new heading line. The problem was, when I tried to open the dataframe, pandas threw the 'empty data error'.
Examination of the source and target files showed them to be perfect, could be opened in Notepad or Excel and all the answers I could find referred to checking file paths, delimiters, encoding, etc.
It seems to me that python doesn't follow our line-by-line instructions but goes off to do other bits while earlier instructions have not yet been completed - multitasking.
To prove my point, I commented out the lines to write to the new file (it had already been created on a previous run) and the df came up prefectly.
                encoding! sometimes small stupid things waste a lot of time. wonder why encoding is not auto detected by pandas!
– Muhammad Moiz Ahmed
                Apr 23, 2020 at 16:43
pd.read_csv(filename)
Will cause the above error. Make sure you f.close() before trying to read your file.
Corrected code:
f = open(filename, 'w')
#some code
f.close()
pd.read_csv(filename)
Great question! Had the same issue when reading multiple .csv files from a directory.
I used the try-except mechanism, which safely skipped the empty files as well as imported the specific error from pandas.errors
Here is a snippet of code that worked for me:
import os
import pandas as pd
from pandas.errors import EmptyDataError
#### get a list of files in data directory
data_dir = "/path/to/dir/"
files_list = os.listdir(data_dir)
#### result containers
good_files = []
bad_files = []
#### iterate files in directory
for one_file in files_list:
    print(f"Parsing: {one_file}")
        #### read csv file to pandas data frame
        df = pd.read_csv(one_file, low_memory=False)
        #### append the file name to a list of good files
        good_files.append(one_file)
        # ...
    except EmptyDataError:
        print(f"No columns to parse from file {one_file}")
        bad_files.append(one_file)
print('Done parsing.')
The output:
> Parsing file: 1.csv
> Parsing file: 2.csv
> No columns to parse from file 3.csv
> Done parsing.
This error will also come if you are reading the csv which you have just created. The solution to this is try creating another thread that will call another function to read csv and perform other operation. The below code will work when you have to merge multiple csv files into one excel file
t4= threading.Thread(function_name)
t4.start()
def function_name():
   lock.acquire()
   writi = ExcelWriter('./Final.xlsx')                                                               
   stock = glob.glob("./*.csv")                                                                                           
   df_file = (pd.read_csv(g) for g in stock)
  for inn, di in enumerate(df_file):
     di.to_excel(writi, sheet_name='view{}.csv'.format(inn)
  writi.save()
  lock.release()
If you are trying to read .txt files into a Pandas Dataframe you would need to have the sep = " " tag. 
This will tell Pandas to use a space as the delimiter instead of the standard comma. 
Also, you if you are importing from a text file and have no column names in the data, you should pass the header=None attribute. Your definition would look like this then:
df = pd.read_csv('output_list.txt', sep=" ", header=None)
                I also have the same problem, but I could find no certain answer. The file is not empty, the white space are really tabs '\t' and there is a header. still I get this pandas.errors.EmptyDataError: No columns to parse from file error as well...
– Behnam
                Feb 1, 2019 at 10:34
        
Thanks for contributing an answer to Stack Overflow!
Please be sure to answer the question. Provide details and share your research!
But avoid …
Asking for help, clarification, or responding to other answers.
Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.