-
Notifications
You must be signed in to change notification settings - Fork 0
/
Copy pathWeek3_TODO.txt
100 lines (80 loc) · 4.82 KB
/
Week3_TODO.txt
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
WEEK 3 TODO:
x 1. Look at what my model is learning! (what are the most predictive words?)
x 2. SQL database with updated data
x 3. Neural net / Skip thoughts??
4. Topic Models
x 5. chrome extension!!
x 6. RNN generator
x 7. web_app front page with explanation
x 8. how to distribute the chrome extension
x 9. test chrome extension with model.
Meeting with Yan -
x 1. Get my app on aws as soon as possible. There might be security issues. Talk to her if I run into this. (https vs http thing)
x 2. Register app on chrome store. Then people can install from the chrome store. She said this was very useful during interviews.
Use hashing vectorizer instead of count vectorizer (if sticking with count)
Threshold. And color items according to how helpful (shades of yellow)
Maybe look at only the code and learn good code vs bad code
Thursday:
x Work on extension. How to keep formatting? Is there a way I can detect code tags/other html??
x Bring the neural net over to the model.
Highlighting based on the predicted helpfulness - more helpful are highlighted in a more saturated yellow
Presentation Notes
1. How to wrap up the arc of your story/pitch
2. what can people now do?
3. mention deliverable up front
4. name on about me
5. maybe put model results on the same plot.slide.
6. emphasize that I treated this as a classifcation problem and why this was important.
7. emphasize that my products works at the sentence level
8. Get more examples for the demonstrates
9. backup slides in case app fails
10. tighten it up!
LINKS TO USE:
http://stackoverflow.com/questions/13784192/creating-an-empty-pandas-dataframe-then-filling-it
Find some other sites to use for this.
http://stackoverflow.com/questions/9236926/concatenating-two-one-dimensional-numpy-arrays - this is where the site works less well
http://stackoverflow.com/questions/21887754/numpy-concatenate-two-arrays-vertically - similar topic but likes all the answers
http://stackoverflow.com/questions/306400/how-do-i-randomly-select-an-item-from-a-list-using-python
http://stackoverflow.com/questions/3501382/checking-whether-a-variable-is-an-integer-or-not
Friday:
x 1. Clean website
x 2. Get everything on github
x 3. Get precision and recall of model
x 4. BUG CHECK!!! (look at the site you know doesn't work - http://stackoverflow.com/questions/6771428/most-efficient-way-to-reverse-a-numpy-array)
Also, I know the site tries to function from the /questions/ page (not a specific question)
5. Use lime to look at what my model likes...
x 6. Change confusion matrix labels (helpful/unhelpful rather than good/bad)
x 7. Titles for figures on website...
x 8. Create readme file(s) for github website. This should mostly be for explaining the code
x 9. Resume!!
This weekend:
1. Improve model
2. javascript addon??
Tonight:
1. Try to combine the good answer finding system and the highlighting systems
2. Maybe look into training my own method for finding sentence breaks...
3. Make sure the linear plots are actually from my linear regression and not from random forest
4. combine roc plots in inkscape
Are there already highlighter chrome extensions??
To run the server as a background app run (from ~/application & insight environment):
gunicorn Web_App:app -D
To turn it off run:
pkill gunicorn
this is from my p2
$ scp [email protected]:/home/ubuntu/nbs/stackex_gru.h5 /home/dan-laptop/Insight/stackex_gru.h5
this is from my t2
$ scp [email protected]:/home/ubuntu/application/Web_App/views.py /home/dan-laptop/Insight/stackex_sum/Web_App/views.py
Work on web_app, input, and explanation page. How to spread the chrome extension?? - the chrome store!!!
https://www.quora.com/Can-I-make-a-Google-Chrome-extension-using-Python - about how to use chrome extension to send website to python hosted flask page.
https://robots.thoughtbot.com/how-to-make-a-chrome-extension
http://charlesleifer.com/blog/a-flask-front-end-and-chrome-extension-for-youtube-dl/ ##this seems like the most helpful thing so far!!!
http://blog.dwyer.co.za/2016/09/reclaiming-web-in-100-lines-of-code.html
https://github.com/aadeyemi/insightdatascience_project
https://github.com/yankou/AirbnbLocal
http://stackoverflow.com/questions/1891738/how-to-modify-current-url-location-in-chrome-via-extensions - update the value of the current tab
https://chrome.google.com/webstore/detail/sifting-the-overflow/japbeffaagcpbjilckaoigpocdgncind?hl=en-US&gl=US
https://tuts.codingo.me/chrome-extension-in-under-60-seconds - this uses some regex
Abayemi Rec -
Do the highlighting in javascript so that I don't have to change people's webpage.
Send the important words vector to javascript. Have javascript to do regex, then insert the class end tag at the end of the found position,
and insert the class start tag at the beginning (class tag is for highlighting)