python - Insert into MongoDB retuns cannot encode object -
i'm doing rather simple insert local mongodb sourced of python pandas dataframe. i'm calling datframe.loc[n].to_dict() , getting dictionary directly df. far until attempt insert, i'm getting 'cannot encode object'. looking @ dict directly showed looked (while writing question) dawned me check each type in dict , found long id number had converted numpy.int64 instead of simple int (which when created dict manually int insert fine).
so, unable find within pandas documentation on adding arguments to_dict allow me override behavior , while there brute force methods fixing issue, there must bit more eloquent way sort issue without resorting sort of thing.
question then, how convert row of dataframe dict insertion mongodb, ensuring using acceptable content types ... or, can further here , use simpler approach each row of dataframe document within mongo?
thanks
as requested, here addendum post sample of data using.
{'account created': 'about 3 hours ago', 'followers': 13, 'following': 499, 'screen name': 'xxxxxxxxxx', 'status': 'alive', 'tweets': 12, 'twitter id': 0000000000l}
this directly to_dict output faulted on insert. copied directly 'test' dict , worked fine. if print out values of each of dicts following...
to_dict = ['alive', 'a_aheref77', 'about 3 hours ago', 12, 13, 499, 0000000000l, objectid('551bd8cfae89e9370851aa64')] test = ['alive', 'xxxxxxxx', 'about 3 hours ago', 499, 13, 12, 0000000000, objectid('551bd6fdae89e9370851aa63')]
the difference (as far can tell) long int, interestingly enough, when did mongo insert shows field being 'number long' within document. hope clarify som.
take @ odo
library. in particular, the mongodb docs. pandas isn't grow kind of to_mongo
methods in near future odo sort of functionality should go. here's example simple dataframe
:
in [13]: import pandas pd in [14]: odo import odo in [15]: df = pd.dataframe({'a': [1, 2, 3], 'b': list('abc')}) in [17]: m = odo(df, 'mongodb://localhost/db::t') in [18]: list(m.find()) out[18]: [{u'_id': objectid('551bfb20362e696200d568d9'), u'a': 1, u'b': u'a'}, {u'_id': objectid('551bfb20362e696200d568da'), u'a': 2, u'b': u'b'}, {u'_id': objectid('551bfb20362e696200d568db'), u'a': 3, u'b': u'c'}]
you can required deps , odo doing
conda install odo pymongo --channel blaze
or
pip install odo
Comments
Post a Comment