UnicodeDecodeError: 'utf-8'编解码器无法解码字节0x8b,位置在第1个字节处:在pandas中读取csv文件时出现无效的起始字节。
UnicodeDecodeError: 'utf-8'编解码器无法解码字节0x8b,位置在第1个字节处:在pandas中读取csv文件时出现无效的起始字节。
我知道类似的问题已经被提出,我已经看过所有的解答并尝试过,但效果不大。我正在使用OSX 10.11 El Capitan操作系统,python3.6版本,使用了虚拟环境,但也尝试过不使用虚拟环境。我正在使用jupyter notebook和spyder3。
我是Python的新手,但了解基本的机器学习知识,并且正在参考一篇博文学习如何解决Kaggle挑战:博文链接,数据集链接。
我在代码的前几行卡住了。
`
import pandas as pd destinations = pd.read_csv("destinations.csv") test = pd.read_csv("test.csv") train = pd.read_csv("train.csv")
但是出现了错误
UnicodeDecodeError Traceback (most recent call last)in () 1 import pandas as pd ----> 2 df = pd.read_csv('destinations.csv', compression='infer',date_parser=True, usecols=([0,1,3])) 3 df.head() /usr/local/lib/python3.6/site-packages/pandas/io/parsers.py in parser_f(filepath_or_buffer, sep, delimiter, header, names, index_col, usecols, squeeze, prefix, mangle_dupe_cols, dtype, engine, converters, true_values, false_values, skipinitialspace, skiprows, nrows, na_values, keep_default_na, na_filter, verbose, skip_blank_lines, parse_dates, infer_datetime_format, keep_date_col, date_parser, dayfirst, iterator, chunksize, compression, thousands, decimal, lineterminator, quotechar, quoting, escapechar, comment, encoding, dialect, tupleize_cols, error_bad_lines, warn_bad_lines, skipfooter, skip_footer, doublequote, delim_whitespace, as_recarray, compact_ints, use_unsigned, low_memory, buffer_lines, memory_map, float_precision) 653 skip_blank_lines=skip_blank_lines) 654 --> 655 return _read(filepath_or_buffer, kwds) 656 657 parser_f.__name__ = name /usr/local/lib/python3.6/site-packages/pandas/io/parsers.py in _read(filepath_or_buffer, kwds) 403 404 # Create the parser. --> 405 parser = TextFileReader(filepath_or_buffer, **kwds) 406 407 if chunksize or iterator: /usr/local/lib/python3.6/site-packages/pandas/io/parsers.py in __init__(self, f, engine, **kwds) 762 self.options['has_index_names'] = kwds['has_index_names'] 763 --> 764 self._make_engine(self.engine) 765 766 def close(self): /usr/local/lib/python3.6/site-packages/pandas/io/parsers.py in _make_engine(self, engine) 983 def _make_engine(self, engine='c'): 984 if engine == 'c': --> 985 self._engine = CParserWrapper(self.f, **self.options) 986 else: 987 if engine == 'python': /usr/local/lib/python3.6/site-packages/pandas/io/parsers.py in __init__(self, src, **kwds) 1603 kwds['allow_leading_cols'] = self.index_col is not False 1604 -> 1605 self._reader = parsers.TextReader(src, **kwds) 1606 1607 # XXX pandas/_libs/parsers.pyx in pandas._libs.parsers.TextReader.__cinit__ (pandas/_libs/parsers.c:6175)() pandas/_libs/parsers.pyx in pandas._libs.parsers.TextReader._get_header (pandas/_libs/parsers.c:9691)() UnicodeDecodeError: 'utf-8' codec can't decode byte 0x8b in position 1: invalid start byte
stackoverflow上的一些答案表明这是因为文件被gzip压缩,但是Chrome下载的是.csv文件,没有看到.csv.gz,并返回文件未找到错误。
然后我读到使用encoding='latin1'
,但是这样做后我得到了解析器错误:
--------------------------------------------------------------------------- ParserError Traceback (most recent call last)in () 1 import pandas as pd 2 ----> 3 destinations = pd.read_csv("destinations.csv",encoding='latin1') 4 test = pd.read_csv("test.csv") 5 train = pd.read_csv("train.csv") /usr/local/lib/python3.6/site-packages/pandas/io/parsers.py in parser_f(filepath_or_buffer, sep, delimiter, header, names, index_col, usecols, squeeze, prefix, mangle_dupe_cols, dtype, engine, converters, true_values, false_values, skipinitialspace, skiprows, nrows, na_values, keep_default_na, na_filter, verbose, skip_blank_lines, parse_dates, infer_datetime_format, keep_date_col, date_parser, dayfirst, iterator, chunksize, compression, thousands, decimal, lineterminator, quotechar, quoting, escapechar, comment, encoding, dialect, tupleize_cols, error_bad_lines, warn_bad_lines, skipfooter, skip_footer, doublequote, delim_whitespace, as_recarray, compact_ints, use_unsigned, low_memory, buffer_lines, memory_map, float_precision) 653 skip_blank_lines=skip_blank_lines) 654 --> 655 return _read(filepath_or_buffer, kwds) 656 657 parser_f.__name__ = name /usr/local/lib/python3.6/site-packages/pandas/io/parsers.py in _read(filepath_or_buffer, kwds) 409 410 try: --> 411 data = parser.read(nrows) 412 finally: 413 parser.close() /usr/local/lib/python3.6/site-packages/pandas/io/parsers.py in read(self, nrows) 1003 raise ValueError('skipfooter not supported for iteration') 1004 -> 1005 ret = self._engine.read(nrows) 1006 1007 if self.options.get('as_recarray'): /usr/local/lib/python3.6/site-packages/pandas/io/parsers.py in read(self, nrows) 1746 def read(self, nrows=None): 1747 try: -> 1748 data = self._reader.read(nrows) 1749 except StopIteration: 1750 if self._first_chunk: pandas/_libs/parsers.pyx in pandas._libs.parsers.TextReader.read (pandas/_libs/parsers.c:10862)() pandas/_libs/parsers.pyx in pandas._libs.parsers.TextReader._read_low_memory (pandas/_libs/parsers.c:11138)() pandas/_libs/parsers.pyx in pandas._libs.parsers.TextReader._read_rows (pandas/_libs/parsers.c:11884)() pandas/_libs/parsers.pyx in pandas._libs.parsers.TextReader._tokenize_rows (pandas/_libs/parsers.c:11755)() pandas/_libs/parsers.pyx in pandas._libs.parsers.raise_parser_error (pandas/_libs/parsers.c:28765)() ParserError: Error tokenizing data. C error: Expected 2 fields in line 11, saw 3
我花了几个小时来调试这个问题,尝试在Atom上打开csv文件(没有其他应用程序能够打开它),在线网站应用程序(一些崩溃了),但没有帮助。我还尝试使用其他人解决问题的内核,但也没有帮助。