Tweetalysis Blog

• Going to try to get Tweetalysis to collect again, there have been phirhose updates so I'll check those out 10:42:29
• "Object of class stdClass could not be converted to string" , such a helpful error 11:05:18
• thinking geo values are messing up my inserts, trying to fix that. 11:13:53
• might have fixed it, inserting 0 for geo information right now, will wait for next hourly insert, running into duplicate entry issue atm 11:18:57
• Now getting duplicate key entry "0" 12:26:36

Just deleted some spam comments, some of the comments were so odd.  Some try to be crafty…others apparently random strings of text.

• Killing the project for a while, when things settle down I hope to turn it back on and play with it more…until then 20:45:43

• @sansta Sorry I missed your messages, I'm not sure you'll find the paper very interesting.It's not technical, just a summary of my work. in reply to sansta 12:24:29
• For some reason SQL relational features are disabled for the moment and my processing script is failing, hoping it's on host's end 12:25:42

• Revising paper today, hoping to finish it for submission tomorrow. 11:26:17
• Nearly done with the paper, I will submit it tomorrow. 19:56:33

• Continuing work on the final paper, mostly fleshed out just needs revision. 18:27:09

• Working on the final paper for the project now that my finals are done. 09:32:09

• Just discovered http://tweetsentiments.com/ they are doing some cool stuff! 12:33:23

• To my surprise the collect script appears to have been running for several days without interruption upon a check of the logs this morning 08:19:15
• bigram_naivebayes test underway. 08:22:50
• 91% on bigram_naivebayes but it didn't predict a single one to be negative oO 08:26:50
• POSbigram_naivebayes up next. 08:27:54
• POSbigram_naivebayes got 76% accuracy, marking a lot negative that are questionable and are making me rethink my classification. 12:23:09
• My focus are negative messages about the brand @starbucks but that might be too hard, some of these are mostly negative but end positive… 12:24:41
• …positive for @starbucks. eg."day sucked, it's rainy, and I'm angry but at least I have starbucks" Negative, but not towards the brand 12:25:58
• Tried the POSBigram_Naivebayes model with a 2nd test set,and it got in the range of 80-90% accuracy. I bet a bigger sample will help improve 19:21:24

• Making a new training set to try to get a better model. 10:34:28
• This is one of the hardest tweets I've had to classify today, "mmmm… the only thing Starbucks is good for… Hot Chocolate…" neg or not? 11:36:04
• Now have 2000 lines for training set and a few hundred for testing. First test is Unigrams_NaiveBayes 12:56:22
• So the unigram_naivebayes model got 93/100, but definitely going to try tweaking the model.Some should clearly have been labeled differently 13:09:43
• Making a unigram_SMO model right now, taking a lot longer to compute than the NaiveBayes did… 14:56:41
• Unigram_SMO is giving me a confidence rating of .666 for every line, and is not performing very well. 15:02:59