-
Notifications
You must be signed in to change notification settings - Fork 26
/
Systems and Programs.txt
2024 lines (1493 loc) · 74.6 KB
/
Systems and Programs.txt
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
###########
### general
# List of Programming Resources and Books
http://resrc.io/list/10/list-of-free-programming-books/#algorithms--datastructures
http://resrc.io/list/10/list-of-free-programming-books/#python
# 100 Most Influential Books According to Stack Overflow
http://cspray.github.io/2012/02/13/a-community-gone-mad.html
http://cspray.github.io/my.so-archive/100-most-influential-programming-books.html
# online interpreters
http://repl.it/languages/
#########
### tools
# ScreenToGif record comp screen and save as animated Gif
https://screentogif.codeplex.com/
http://www.discourse.org/
#########
### computer science artificial intelligence machine learning non-finance
# Prisma program neural network can redraw pictures in the style of paintings by well-known artists
http://www.bloomberg.com/view/articles/2016-06-20/new-masters-software-has-painterly-technique-down-pat
http://phys.org/news/2016-06-russian-photo-app-prisma-splash.html
The Prisma neural network can separate the style and technique of a painting from its content (people, trees, etc.)
The Prisma program is able to paint a given person's face in the style of Picasso or Van Gogh.
Prisma was written by the Russian programer Alexei Moiseenkov, using research from German universities.
The Prisma program has reached the no. 1 spot in the iOS app store.
# University of Pennsylvania statistician Richard Berk created machine learning algorithm that predicts likelihood someone will commit domestic violence crimes
http://www.bloombergview.com/articles/2016-03-08/the-crime-you-have-not-yet-committed
The algorithm uses data such as age, sex, zip code, age at first arrest, list of previous charges like drunk driving, animal mistreatment, and firearms crimes.
The algorithm doesn't use race directly but race can be inferred from other data like zip code.
The algorithm outperforms parole judges, with about 20% of those released by judges were later arrested, while only 10% of the computer’s choices were arrested.
Philadelphia uses the algorithm to assign parolees to high, medium, and low-risk groups, allowing parole officers to focus most of their attention on the high-risk cases.
# machine learning classification algorithms are easily fooled by adversarial examples
Szegedy ANN Deep Learning Adversarial Misclassification.pdf
http://www.kdnuggets.com/2015/07/deep-learning-adversarial-examples-misconceptions.html
http://www.kdnuggets.com/2014/06/deep-learning-deep-flaws.html
Adversarial examples are images (data sets) that are slightly modified in such a way that machine learning classification algorithms are fooled into classifying them incorrectly.
Adversarial examples are often indistinguishible by the human eye from the original images, yet are incorrectly classified by machine learning algorithms.
The human eye (perception) is not susceptible to adversarial examples that are able to fool machine learning algorithms.
Deep learning is not the only machine learning algorithm susceptible to adversarial examples, and it's not more susceptible than other algorithms.
Rubbish examples are random images that algorithms classify to some class with high confidence, even though they should not belong to any class.
Machine learning algorithms often classify a very large percentage of random images to some class with high confidence, even though they should not belong to any class.
The mis-classification of adversarial examples by machine learning algorithms may be caused by the linear nature of the algorithms.
Machine learning algorithms classify images (data sets) in a very different way than human (animal) perception does.
Human (animal) perception performs abstractions of images by identifying edges of objects, and by identifying objects contained in images.
Human (animal) perception classifies images based on the objects (elements) they contain.
A face for example has two eyes, a nose and a mouth.
But edge detection is a nonlinear process, so linear algorithms can't identify the edges of objects.
Machine learning classification algorithms don't create abstractions of images (data sets), the way human (animal) perception does.
Instead, algorithms classify images based on pixels, so they're susceptible to small changes of individual pixels.
# Intelligent learning systems are composed of two complementary sub-systems
http://techxplore.com/news/2016-06-insights-human-foster-smarter-artificial.html
The first system acquires knowledge and skills from exposure to experiences, while the second system stores and organizes the experiences so that they can be replayed, and integrated back into the first system.
In animal brains the first system is the frontal cortex, and the second system is the hippocampus.
# Defective lists the major defects of C++
http://yosefk.com/c++fqa/defective.html
# Ray Solomonoff invented algorithmic probability related to Kolmogorov complexity and algorithmic information theory
https://en.wikipedia.org/wiki/Ray_Solomonoff
Algorithmic probability can predict the most likely next event in a series of events, using inductive inference (Bayes' rule).
# Laszlo Babai claims to have proven that P=NP
http://www.chicagomag.com/city-life/October-2015/Why-Computer-Scientists-and-Mathematicians-Are-Stunned-By-a-Chicago-Professors-New-Proof/
That means that every problem whose solution can be quickly verified,
can also be quickly solved.
# Does P equal NP? problem
http://news.mit.edu/2009/explainer-pnp
Does every problem whose solution can be quickly verified, can also be quickly solved?
P-problems have solutions that are polynomial in number of elements.
For example sorting a list is a P-problem.
NP-problems have solutions that are non-polynomial (exponential) in number of elements.
Some of them may also have solutions in polynomial time, but nobody knows if they do.
For example finding the prime factors of a number is a NP-problem.
Verifying a solution is a forward process (for example multiplication).
Finding a solution is an inverse process (for example division or integration).
"Does P equal NP?" means that if a solution can be verified in polynomial time,
then can the solution also be found in polynomial time?
In other words, can efficient solutions to NP-problems ever be found?
# Italian computer scientist Massimo Marchiori invented Hyper Search algorithm inspiration for Google's PageRank
http://www.bloomberg.com/news/2014-05-15/meet-the-italian-man-who-beat-google-to-web-search-and-gave-it-away-for-free-.html
# Exquisite Forest algo generates music
http://labs.dinahmoe.com/theme/
http://www.exquisiteforest.com/
# Tate
http://milk.co/forest-tate
# machine learning boom of 2015
http://www.bloomberg.com/news/articles/2015-12-08/why-2015-was-a-breakthrough-year-in-artificial-intelligence
# Google RankBrain machine learning system trying to understand language
http://www.bloomberg.com/news/articles/2015-10-26/google-turning-its-lucrative-web-search-over-to-ai-machines
RankBrain parses text on websites, and ranks websites nased on their relevance for certain queries.
# NEAT algorithm for evolving neural networks and simulating creative processes
http://fivethirtyeight.com/features/stop-trying-to-be-creative/
Creative processes are driven by a hazy intuition or vision, instead of a plan.
Creative processes are deliberate, not just random searches, and rely on serendipity.
Creative processes can't be planned or predicted ahead of time because they don't have a well defined goal, so the time to arrive at their objective can't be known in advance.
Pursuing creative solutions instead of well defined goals is therefore risky, since many attempts will not pay off, but the potential payoffs are higher as well.
# evolve pictures using genetic programming
http://picbreeder.org/
# machine learning boom funded by startups
http://www.bloomberg.com/news/articles/2015-02-03/i-ll-be-back-the-return-of-artificial-intelligence
Most AI startups aren’t developing machines capable of animal intelligence, but are instead making clever tools to solve practical data analysis problems.
The extraordinary increase in chip performance over the last 17 years has allowed applications of machine learning software.
Video chips in 2014 became 84.3 times more powerful than those in 2004.
# Hofstadter The Man Who Would Teach Machines to Think
http://www.theatlantic.com/magazine/archive/2013/11/the-man-who-would-teach-machines-to-think/309529/
AI today has become too much like the man who tries to get to the moon by climbing a tree: One can report steady progress, all the way to the top of the tree.
# Is A Simulated Brain Conscious?
http://www.popsci.com/article/science/simulated-brain-conscious
# Human brains combined with computers
http://online.wsj.com/articles/a-lesson-from-alan-turing-how-creativity-drives-machines-1411749814
Ada Lovelace envisioned that the combined talents of humans and computers will indefinitely be more creative than any computer working alone.
Licklider wrote in 1960: Human brains and computing machines will be coupled together very tightly, and the resulting partnership will think as no human brain has ever thought and process data in a way not approached by the information-handling machines we know today.
# machine learning experts in high demand
http://www.wsj.com/article_email/artificial-intelligence-experts-are-in-high-demand-1430472782-lMyQjAxMTA1MzA2MTMwNTEyWj
# kaggle: selecting right data much more important than sophisticated models
http://www.nytimes.com/2015/03/22/opinion/sunday/making-march-madness-easy.html
Ensemble prediction algorithms often do better than their components do alone.
# machine learning at Google
http://www.wired.com/2014/07/google_brain/
http://www.wired.com/2013/12/facebook-yann-lecun-qa/
Deep learning was started by Geoff Hinton, Yann LeCun, and Yoshua Bengio, from the University of Montreal.
They got funding from the CIFAR foundation (Canadian Institute For Advanced Research).
Geoff was the director, and Yann LeCun was the chair of the advisory committee, and they would meet twice a year to discuss progress.
# Machine Learning Most Popular Course At Stanford
http://www.forbes.com/sites/anthonykosner/2013/12/29/why-is-machine-learning-cs-229-the-most-popular-course-at-stanford/
# machine learning replacing human workers
http://www.bloomberg.com/news/2014-03-12/your-job-taught-to-machines-puts-half-u-s-work-at-risk.html
Three associates selected relevant documents from a smaller sample, "teaching" their reasoning to the computer. The software’s algorithms then sorted the remaining material by importance.
http://www.bloomberg.com/infographics/2014-03-12/job-automation-threatens-workforce.html
# machine learning scores consumer credit
http://www.bloomberg.com/news/2014-10-01/lender-charging-390-uses-data-to-screen-out-deadbeats.html
An algorithm called Naive Bayes checks whether individual traits such as age or marital status help predict defaults.
Random forests place borrowers in groups and looks for patterns to emerge.
Hidden Markov models analyze whether events such as lapsed mobile-phone payments, signal an unseen condition such as illness.
# robot learning through crowdsourcing imitation, demonstration
http://www.kurzweilai.net/crowdsourcing-for-robots
http://www.washington.edu/news/2014/06/26/ask-the-crowd-robots-learn-faster-better-with-online-helpers/
http://news.nationalgeographic.com/news/2013/07/130719-robot-lfd-pr2-artificial-intelligence-crowdsourcing-robotics-machine-learning/
http://www.scienceworldreport.com/articles/15704/20140627/robots-learn-faster-with-online-input.htm
# humans teaching computers: Mechanical Turk online computer training
https://www.mturk.com/mturk/welcome
# Robots That Learn Through Repetition, Not Programming
http://www.technologyreview.com/news/530871/robots-that-learn-through-repetition-not-programming/
# robots that collaborate with people
http://www.kurzweilai.net/developing-robots-that-collaborate-with-people
http://www.economist.com/news/technology-quarterly/21584455-robotics-new-breed-robots-being-designed-collaborate-humans
http://www.technologyreview.com/news/530696/how-human-robot-teamwork-will-upend-manufacturing/
http://news.discovery.com/tech/robotics/robot-human-crosstraining-130218.htm
# machine learning began dominating traditional AI
http://www.theatlantic.com/magazine/archive/2013/11/the-man-who-would-teach-machines-to-think/309529/
# Artificial Intelligence programs write news articles from data - automated narrative generation
http://www.nytimes.com/2015/03/08/opinion/sunday/if-an-algorithm-wrote-this-how-would-you-even-know.html
Artificial Intelligence programs identify patterns, trends, and anomalies, and then create explanatory narrative.
The Associated Press uses Automated Insights’ Wordsmith platform to create more than 3,000 financial reports per quarter.
Kristian Hammond estimates that 90% of news could be algorithmically generated by the mid-2020s
# Narrative Science programs write sports and finance articles
http://www.wired.com/2012/04/can-an-algorithm-write-a-better-news-story-than-a-human-reporter/
Kristian Hammond co-founded Narrative Science.
Human meta-writers compile the vocabulary and create a framework for articles.
Clients customize the tone of the stories.
# Philip Parker AI programs wrote over 1 Million Books
http://www.huffingtonpost.com/2013/02/11/philip-parker-books_n_2648820.html
http://www.businessinsider.com/novels-written-by-computers-2014-11
http://www.narrativescience.com/quill
# Robot Journalists: people can't tell apart sports stories written by computers from those written by human writers
http://www.tandfonline.com/doi/pdf/10.1080/17512786.2014.883116
# Benjamin Bratton: AI has little to do with human or animal intelligence
http://opinionator.blogs.nytimes.com/2015/02/23/outing-a-i-beyond-the-turing-test/
Why define an advanced A.I. by its resemblance to ours?
Human intelligence simply can’t exhaust the possibilities of intelligence.
We need a view of A.I. that is less narcissistic, more than simply a machine version of ourselves.
The criteria for being "intelligent" doesn’t have have to be how it reflects humanness back at us.
Biomorphic imitation is not how we design complex technology.
Airplanes don’t fly like birds fly,
Whether airplanes fly like birds isn't a test to determine if planes can "really" fly.
Programming A.I. not to "harm humans" (inspired by Isaac Asimov) makes sense only when an A.I. knows what humans are and what harming them means.
Even if A.I. has no malevolence towards us, it might harm us by following exactly our well-meaning instructions to an idiotic and catastrophic extreme.
# The Second Machine Age
http://www.mckinsey.com/Insights/Strategy/Artificial_intelligence_meets_the_C-suite
The Industrial Revolution was when humans overcame the limitations of our muscle power.
# The Steely, Headless King of Texas Hold ’Em
http://www.nytimes.com/2013/09/08/magazine/poker-computer.html
#########
### blogs
# Zygmunt Zajac economist and machine learning expert
http://fastml.com/
https://github.com/zygmuntz
# Pawel Lachowicz Python quant
http://www.quantatrisk.com/
# Grzegorz Chrupala computational linguist and machine learning expert
http://grzegorz.chrupala.me/
# Triskelion kaggle Master and machine learning expert
http://mlwave.com/
http://datalligence.blogspot.com/
# Gabriel Synnaeve kaggle Master from LSCP and INRIA
http://snippyhollow.github.io/
http://snippyhollow.github.io/blog/
# good links and papers
http://snippyhollow.github.io/blog/2014/08/09/so-you-wanna-try-deep-learning/
# simple Python deep neural network with or w/o dropout
https://gist.github.com/SnippyHolloW/8a0f820261926e2f41cc
# Carl Vogel - converting R to Python - NY economist, quant, HF finance data
http://slendermeans.org/
http://slendermeans.org/category/will-it-python.html
https://github.com/carljv/Will_it_Python
# Scott Fortmann-Roe: out of sample prediction error
http://scott.fortmann-roe.com/docs/MeasuringError.html
# Scott Fortmann-Roe: bias-variance tradeoff
http://scott.fortmann-roe.com/docs/BiasVariance.html
# Nathan Yau - FlowingData data exploration, analysis, and visualization
http://flowingdata.com/
# stackexchange blogs
http://stackexchange.com/blogs
http://stats.blogoverflow.com/
http://security.blogoverflow.com/
# Jason Bryer: R, Jekyll, web publishing plus multiple R packages
http://jason.bryer.org/
http://jbryer.github.com
https://github.com/jbryer
# github hosted page setup
https://pages.github.com/
# website modified from Skeleton, hosted on Github Pages using Jekyll
http://jason.bryer.org/about.html
# Using (R) Markdown, Jekyll, & GitHub for a Website
http://jason.bryer.org/posts/2012-12-10/Markdown_Jekyll_R_for_Blogging.html
http://www.getskeleton.com/
#########
### education
# New York R Conference April 8-9, 2016
http://www.rstats.nyc/index.html
# ScriptScoop aggregator of code examples
http://www.scriptscoop.com/
# StackOverflow flaws
http://michael.richter.name/blogs/why-i-no-longer-contribute-to-stackoverflow
# Knewton personalized online courses, figure out what each student knows and how that student learns best, then recommends what to study next.
http://www.knewton.com/
# DataCamp & rcademy - Jonathan Cornelissen (works with Boudt)
https://www.datacamp.com/
# data science projects and instruction
http://www.teamleada.com/
# Software Carpentry sponsored by RStudio: volunteer instructors for teaching researchers computing skills
http://software-carpentry.org/
http://software-carpentry.org/blog/
# Caltech machine learning online courses
https://work.caltech.edu/telecourse.html
https://www.coursera.org/course/ml
# Metacademy graphs of knowledge dependency by John Langford, Roger Grosse, and Colorado Reed
http://metacademy.org/
http://metacademy.org/graphs/concepts/feed_forward_neural_nets
http://metacademy.org/roadmaps/
# username (public): algoquant
# email (not public): algoquant@algoquants.ch
http://hunch.net/?p=2714
http://www.cs.toronto.edu/~rgrosse/
http://obphio.us/
# online courses
https://www.edx.org/
https://www.udacity.com/
# MOOC with interactive content by David Cox Professor of Molecular and Cellular Biology and Computer Science
http://www.mcb80x.org/
# thinkful online mentoring course Python, Ruby, AngularJS
http://www.thinkful.com/
# Matrix Reference Manual
http://www.ee.ic.ac.uk/hp/staff/dmb/matrix/intro.html
# C++ introduction
https://tfetimes.com/beginners-guide-to-c/
https://tfetimes.com/crash-course-in-c/
#########
### social learning
# Dream to Learn social learning platform
https://dreamtolearn.com/
https://dreamtolearn.com/content/about.html
# Ryan Anderson - build a social platform for learning, development and growth
https://dreamtolearn.com/ryan
https://dreamtolearn.com/ryan/data_analytics_viz/71
https://dreamtolearn.com/node/9H0SX6Y7KX740GNSX4IFIBFZW
#########
### machine learning
Machine learning refers to creating statistical models that fit into some set of data. Linear regression is an example of a machine learning model. Machine learning uses several techniques that make it more powerful:
First, it uses cross-validation (aka backtesting), or testing of the model on out-of-sample data.
Second, it uses regularization, or model parameter constraints, to avoid so-called overfitting.
Third, it uses ensemble learning, by combining the results of many models to achieve better performance than any single model.
# good intro for beginners
http://machinelearningmastery.com/
http://machinelearningmastery.com/blog/
http://machinelearningmastery.com/how-to-get-started-with-machine-learning-algorithms-in-r/
# very simple intros
https://medium.com/@ageitgey/machine-learning-is-fun-80ea3ec3c471
http://www.datasciencecentral.com/profiles/blogs/a-tour-of-machine-learning-algorithms
# "Statistical Modeling: The Two Cultures" by Leo Breiman: statistics vs. machine learning
http://brenocon.com/blog/2008/12/statistics-vs-machine-learning-fight/
http://stats.stackexchange.com/questions/6/the-two-cultures-statistics-vs-machine-learning
"The biggest difference is that statistics emphasizes inference, whereas machine learning emphasized prediction."
"In statistics, you infer the process by which the data was generated."
"In machine learning, you predict what future data will look like w.r.t. some variable."
"You can still make good predictions when p >> n, but you can't make very good inferences about what variables are actually important and why."
http://www.datasciencecentral.com/profiles/blogs/17-analytic-disciplines-compared
# Andrew Gelman likes machine learning
http://www.stat.columbia.edu/~gelman/
http://andrewgelman.com/2005/09/23/classparticipat/
# Data Science Toolkit
http://www.datasciencecentral.com/profiles/blogs/the-data-science-toolkit-my-boot-camp-ciriculum
# MXNet is a deep learning framework
http://mxnet.readthedocs.org/en/latest/
https://github.com/dmlc/mxnet
http://mxnet.readthedocs.org/en/latest/R-package/index.html
# Leo Breiman from Berkeley invented random forests
http://www.stat.berkeley.edu/~breiman/RandomForests/
http://www.stat.berkeley.edu/~breiman/sf.html
http://www.stat.berkeley.edu/~breiman/
#########
### machine learning software
# Machine Learning Open source (MLOSS)
http://mloss.org/software/
http://mloss.org/community/
# Machine Learning Open source
# MLOSS is run by:
http://sonnenburgs.de/soeren/category/
http://blog.mikiobraun.de/
http://www.ong-home.my/
# RapidMiner environment for machine learning, data mining, and predictive analytics
http://rapidminer.com/
http://rapidminer.com/rapidminer-and-r/
http://www.aphysicistinwallstreet.com/2010/11/example-rapidminer-r-for-trading.html
# R and RapidMiner for time series forecasting
http://www.simafore.com/blog/bid/204923/Combining-power-of-R-and-RapidMiner-for-time-series-forecasting
http://www.kdnuggets.com/2010/11/rapidminer-r-extension.html
### Shogun machine learning toolbox, Support Vector Machines (SVM)
# from Berlin Institute of Technology
http://www.shogun-toolbox.org/
http://www.shogun-toolbox.org/doc/en/3.0.0/index.html
http://en.wikipedia.org/wiki/Shogun_(toolbox)
https://github.com/shogun-toolbox/shogun/
http://sonnenburgs.de/soeren/category/
https://github.com/lisitsyn
https://github.com/vigsterkr
# R Static Interface Examples
http://www.shogun-toolbox.org/doc/en/3.0.0/r_static_examples.html
# R Modular Interface Examples
http://www.shogun-toolbox.org/doc/en/3.0.0/r_modular_examples.html
# scikit-learn by INRIA David Cournapeau
http://scikit-learn.org/stable/index.html
http://en.wikipedia.org/wiki/Scikit-learn
# scikit-learn by INRIA Olivier Grisel
http://ogrisel.com/
# Machine Learning in Python - Trento
http://mlpy.sourceforge.net/
# Support Vector Machine Python
http://www.yaksis.com/posts/why-use-svm.html
# Machine Learning on cloud
https://bigml.com/
http://en.wikioffuture.org/BigML/
#########
### neural networks
# Hacker's guide to Neural Networks
http://karpathy.github.io/neuralnets/
# Andrej Karpathy: Recurrent Neural Networks
http://karpathy.github.io/2015/05/21/rnn-effectiveness/
https://cartesianfaith.com/2016/02/23/is-deep-learning-a-markov-chain-in-disguise/
http://cs.stanford.edu/people/karpathy/
# Michael Nielsen book on artificial neural networks and deep learning
http://michaelnielsen.org/blog/michael-a-nielsen/
http://michaelnielsen.org/blog/
http://www.michaelnielsen.org/ddi/
https://github.com/mnielsen
# Reinventing explanation
http://michaelnielsen.org/reinventing_explanation/index.html
# Reinventing Discovery book on collective intelligence and open science
http://www.amazon.com/gp/product/0691148902
# Neural Networks course Michael Biehl
http://www.cs.rug.nl/~biehl/Teaching/NN/index.html
# Pattern Recognition course Michael Biehl
http://www.cs.rug.nl/~biehl/Teaching/PR/
# Toronto ANN course
https://www.coursera.org/course/neuralnets
# Data Science Toolbox environment in Ubuntu Linux includes Python and R
http://datasciencetoolbox.org/
https://github.com/DataScienceToolbox/data-science-toolbox/
http://jeroenjanssens.com/2013/12/07/lean-mean-data-science-machine.html
# GraphLab Create machine learning Python API system
http://graphlab.com/
https://dato.com/
https://github.com/graphlab-code/graphlab
# John Langford: Machine Learning Theory
http://hunch.net/
# Metacademy: a package manager for knowledge
http://hunch.net/?p=2714
http://metacademy.org/roadmaps/rgrosse/deep_learning
# Structural Risk Minimization
http://hunch.net/~jl/projects/prediction_bounds/thesis/mathml/thesisse19.xml
# Avoid Machine Learning Mistakes article
https://medium.com/@nomadic_mind/new-to-machine-learning-avoid-these-three-mistakes-73258b3848a4
# Kevin P. Murphy book Machine Learning: Probabilistic Perspective with Matlab code
http://www.amazon.com/Machine-Learning-Probabilistic-Perspective-Computation/dp/0262018020
http://www.cs.ubc.ca/~murphyk/MLbook/
https://github.com/probml
#########
### deep learning
# getting started
http://snippyhollow.github.io/blog/2014/08/09/so-you-wanna-try-deep-learning/
https://gist.github.com/SnippyHolloW/8a0f820261926e2f41cc
https://github.com/SnippyHolloW
http://deeplearning.net/
# Deep Learning: Courses, Tutorials, Software
http://www.kdnuggets.com/2014/05/learn-deep-learning-courses-tutorials-overviews.html
# primer on deep learning
http://www.datarobot.com/blog/a-primer-on-deep-learning/
# Unsupervised Feature Learning and Deep Learning (UFLDL) Tutorial
http://deeplearning.stanford.edu/wiki/index.php/UFLDL_Tutorial
# interview with Yann LeCun of NYU
# Deep Learning was developed by Geoff Hinton (Toronto, Google), Yann LeCun (NYU, Facebook), and Yoshua Bengio (Montreal).
http://www.wired.com/2013/12/facebook-yann-lecun-qa/
http://www.kdnuggets.com/2014/02/exclusive-yann-lecun-deep-learning-facebook-ai-lab.html
# Deep learning two main packages: Torch7, and Theano + Pylearn2
# Torch7 is an extension of the LuaJIT language (compiled version of Lua language), which is extremely fast.
# Theano+Pylearn2 has the disadvantage of using Python (it's slow).
# deep learning sparse filtering Kaggle: Black Box Challenge
http://cs.stanford.edu/~jngiam/
http://fastml.com/deep-learning-made-easy/
# Deep Learning and the Renormalization Group Works
https://www.linkedin.com/profile/view?id=1259300
https://charlesmartin14.wordpress.com/2015/03/25/why-does-deep-learning-work/
https://charlesmartin14.wordpress.com/2015/04/01/why-deep-learning-works-ii-the-renormalization-group/
# Hannes Bretschneider: PhD Student Neural networks & Deep Learning University of Toronto
https://ha.nnes.io/
https://github.com/hannes-brt
# GPU Deep Learning Library in Python
https://github.com/hannes-brt/hebel
# commercial Numba to program the GPU from Python
http://continuum.io/webinars/using-numba-program-gpu
https://store.continuum.io/cshop/accelerate/
https://developer.nvidia.com/how-to-cuda-python
http://devblogs.nvidia.com/parallelforall/numbapro-high-performance-python-cuda-acceleration/
# Running Python on GPUs
http://fastml.com/running-things-on-a-gpu/
# Machine Learning consultants and examples (Matlab, Python, VBA)
http://www.sitmo.com/
# Artificial Intelligence: A Modern Approach
http://aima.cs.berkeley.edu/
http://bayesia.us/
### articles
# deep learning degrades quickly as art becomes more abstract
https://medium.com/the-physics-arxiv-blog/computer-vision-algorithms-detect-human-figures-in-cubist-art-e82995bb42a0
#########
### machine learning cloud
# H2O machine learning with R API - Matt Dowle
http://www.h2o.ai/
http://www.h2o.ai/docs/
http://www.h2o.ai/download/h2o/r
http://www.h2o.ai/use-cases/predictive-modeling-factories/
# H2O University
http://university.h2o.ai/?_ga=1.199135535.147602298.1461338879
# H2O prediction engine, sponsored by Stephen Boyd, Rob Tibshirani, Trevor Hastie
http://0xdata.com/
http://0xdata.com/downloadtable/
http://docs.0xdata.com/
https://github.com/0xdata
http://0xdata.github.io/h2o/
http://0xdata.com/blog/2014/06/h2o-killer-application-spark/
# yhat Python and R model hosting
http://www.yhathq.com/
http://blog.yhathq.com/
https://github.com/yhat
# random forests in R and Python hosted on yhat
http://blog.yhathq.com/posts/comparing-random-forests-in-python-and-r.html
# yhat Sciencebox R and Python server with RStudio
https://yhathq.com/products/sciencebox
http://venturebeat.com/2014/06/17/yhat-sciencebox/
# PredictionIO with either PHP, Python, or Ruby code
http://prediction.io/
https://github.com/PredictionIO
http://docs.prediction.io/current/
# Machine Learning on cloud
https://bigml.com/
http://en.wikioffuture.org/BigML/
# Machine Learning for marketing
http://www.wise.io/
# cloud ML using fora functional language
http://www.ufora.com/
http://en.wikibooks.org/wiki/FORA/tutorial
FORA programs dynamically scale in parallell to use all the computer resources available to minimise runtime.
FORA interpreter dynamically compiles code so that execution efficiency is similar to a program written in C.
#########
### kaggle ML prediction competitions
# kaggle ML prediction competitions
https://www.kaggle.com/
https://www.kaggle.com/wiki/KaggleMemberFAQ
http://blog.kaggle.com/
http://www.forbes.com/sites/parmyolson/2012/09/13/worlds-top-data-scientists-open-doors-to-big-cash-contests/
# DataRobot by Xavier Conort - kaggle Master
http://www.datarobot.com/
http://www.datarobot.com/blog/
https://www.kaggle.com/users/17379/xavier-conort
# kaggle blog
http://blog.kaggle.com/
# Kaggle Masters - how they do it
http://blog.kaggle.com/2014/08/01/learning-from-the-best/
"Friedman’s gradient boosting machine works great" - gbm package
# kaggle masters
https://www.kaggle.com/users
https://www.kaggle.com/wiki/KaggleMemberFAQ
https://www.linkedin.com/pub/owen-zhang/51/aa0/363
http://www.kaggle.com/users/7756/owen
# kaggle Titanic Competition using R
http://www.joyofdata.de/blog/titanic-challenge-kaggle-svms-kernlab-decision-trees-party/
https://www.kaggle.com/users/130117/curt-wehrley
https://www.kaggle.com/c/titanic-gettingStarted/details/new-getting-started-with-r
https://github.com/wehrley/wehrley.github.io/blob/master/SOUPTONUTS.md
# Solution using Random Forests
http://trevorstephens.com/post/72916401642/titanic-getting-started-with-r
https://github.com/trevorstephens/titanic/commit/d2dfe4ef258c9f17efc911f33eaecb2fa3c8faca
# Russian machine learning cloud
http://www.octopuscip.comcastbiz.net/
### Machine Learning courses
# udacity Machine Learning courses
https://www.udacity.com/course/ud675
https://www.udacity.com/course/ud741
https://www.udacity.com/course/ud820
# Stanford Machine Learning course
https://www.coursera.org/course/ml
https://class.stanford.edu/dashboard
https://class.stanford.edu/courses/HumanitiesScience/StatLearning/Winter2014/about
https://class.stanford.edu/courses/HumanitiesScience/StatLearning/Winter2014/wiki/StatLearning/
https://class.stanford.edu/courses/HumanitiesScience/StatLearning/Winter2014/progress
https://class.stanford.edu/courses/HumanitiesScience/StatLearning/Winter2014/courseware/d9820868fd3642f19ee45e273e3dfafa/58bee9a364d14f6d9306f0e554acf46c/
# Stanford Machine Learning course using Octave-Forge
http://openclassroom.stanford.edu/MainFolder/HomePage.php
http://openclassroom.stanford.edu/MainFolder/CoursePage.php?course=MachineLearning
# Pedro Domingos UWash machine learning coursera
http://homes.cs.washington.edu/~pedrod/
https://www.coursera.org/course/machlearning
# IPython Scikit machine learning examples tutorials
http://ipython-books.github.io/featured-04/
http://amueller.github.io/sklearn_tutorial/#/
http://nbviewer.ipython.org/github/gmonce/scikit-learn-book/tree/master/
https://github.com/ipython/ipython/wiki/A-gallery-of-interesting-IPython-Notebooks
http://nbviewer.ipython.org/github/agconti/kaggle-titanic/blob/master/Titanic.ipynb
http://www.datasciencecentral.com/forum/topics/interesting-tutorials-from-pycon-2014-usa?groupUrl=resources
http://nbviewer.ipython.org/github/temporaer/tutorial_ml_gkbionics/tree/master/
http://nbviewer.ipython.org/github/temporaer/tutorial_ml_gkbionics/blob/master/2%20-%20KMeans.ipynb
http://pyvideo.org/video/2561/exploring-machine-learning-with-scikit-learn
http://pyvideo.org/video/2614/diving-deeper-into-machine-learning-with-scikit-l
#########
### Machine Learning resources
# good websites
http://www.datasciencecentral.com/
http://www.analyticbridge.com/
# Machine Learning Resources
http://www.datasciencecentral.com/group/resources/forum/topics/comprehensive-list-of-data-science-resources
http://www.datasciencecentral.com/group/resources/forum/topics/a-large-set-of-machine-learning-resources-for-beginners-to-mavens
# Big data sets available for free
http://www.datasciencecentral.com/group/resources/forum/topics/big-data-sets-available-for-free
# Tutorials from PyCon 2014 – USA
http://www.datasciencecentral.com/group/resources/forum/topics/interesting-tutorials-from-pycon-2014-usa
# how to post your free stuff
http://www.datasciencecentral.com/group/resources/forum/topics/free-stuff-for-data-science-publishers-authors-bloggers-professor
#########
### Python
# Python documentation
https://docs.python.org/3/
### Python learning
# Codecademy Python course
http://www.codecademy.com/tracks/python
# Google Python Class
https://developers.google.com/edu/python/
# Python tutorials
https://docs.python.org/3.5/tutorial/
http://www.learnpython.org/
http://www.diveintopython.net/
# Udacity Intro to Computer Science with Python certificate (pay)
https://www.udacity.com/course/cs101
# Python Coursera certificate (pay)
https://www.coursera.org/course/interactivepython1
# Python O’Reilly School of Technology certificate (pay)
http://www.oreillyschool.com/certificate-programs/python-programming/
# Python Course
http://software-carpentry.org/
# Python learning resources
http://resrc.io/list/10/list-of-free-programming-books/#python
# Python interpreter
http://repl.it/languages/Python
# Anaconda: Python analytics, scientific computing, and data visualization
http://continuum.io/
# install Anaconda: Python + Data Science - Quick Start Guide
http://bigdata.braccialli.net/2014/06/python-for-data-science-quick-start.html
# Python missing data debate
http://www.numpy.org/NA-overview.html
# Python doesn't have missing data types
http://www.activeanalytics.co.uk/blog/rvspythonwhyrisstillthekingofstatisticalcomputing
#########
### IPython interactive computing
http://ipython.org/
# IPython notebooks
http://ipython.org/notebook.html
https://github.com/ipython/ipython/wiki/A-gallery-of-interesting-IPython-Notebooks
# IPython vs knitr, or Python vs R
http://yihui.name/en/2012/11/ipython-vs-knitr/
# share IPython notebooks
http://nbviewer.ipython.org/
# Using R Within the IPython Notebok
http://nbviewer.ipython.org/github/ipython/ipython/blob/master/examples/Builtin%20Extensions/R%20Magics.ipynb
#########
### Python scientific computing
http://www.numpy.org/
http://www.slideshare.net/zukun/scientific-programming-in-python
http://fperez.org/py4science/index.html
http://fperez.org/py4science/starter_kit.html
# SimPy Python for simulations
https://simpy.readthedocs.org/en/latest/
http://en.wikipedia.org/wiki/SimPy
# Python pandas - Wes McKinney
http://pandas.pydata.org/index.html
http://blog.wesmckinney.com/
# Pandas tutorials
http://pandas.pydata.org/pandas-docs/stable/tutorials.html
http://www.bearrelroll.com/2013/05/python-pandas-tutorial/
http://www.gregreda.com/2013/10/26/intro-to-pandas-data-structures/
# data wrangling with Pandas
http://nbviewer.ipython.org/github/fonnesbeck/pytenn2014_tutorial/blob/master/Part%201.%20Data%20Wrangling%20with%20Pandas.ipynb?create=1
# Python for research
http://www.stat.washington.edu/~hoytak/blog/whypython.html
# Python Data Analysis Library
# Python Wakari.IO Web-based Python Data Analysis
https://wakari.io/
### Python discussions
# C++/Python/R alternatives
http://quant.stackexchange.com/questions/728/switching-from-c-to-r-limitations-applications
# Why is SciPy better than Matlab or IDL?
https://groups.google.com/forum/#!topic/scipy-user/DrFRcJWWPJo
# Python is steadily eating other languages’ lunch
http://www.talyarkoni.org/blog/2013/11/18/the-homogenization-of-scientific-computing-or-why-python-is-steadily-eating-other-languages-lunch/
# Top Languages for analytics, data mining, data science
http://www.kdnuggets.com/2013/08/languages-for-analytics-data-mining-data-science.html?goback=%2Enpp_matthew*5dowle%2F8%2F435%2F512%2Eanb_155029_*2_*1_*1_*1_*1_*1#%21
# R and Python
http://www.linkedin.com/groupItem?view=&gid=155029&type=member&item=68240217&qid=60beb1ba-cd0c-477f-bbd3-49d5d5b40bc3&trk=groups_most_popular-0-b-ttl&goback=%2Enpp_matthew*5dowle%2F8%2F435%2F512%2Eanb_155029_*2_*1_*1_*1_*1_*1%2Egmp_155029
# Index Python Packages
https://pypi.python.org/pypi
# Django Python Web framework
https://www.djangoproject.com/
#########
### Python finance
# Python quants
http://tpq.io/
http://pqp.io/
https://www.linkedin.com/pulse/open-source-quantitative-finance-neil-fowler
#########
### Jupyter Notebooks
# tutorial Jupyter Notebooks
https://github.com/michhar/useR2016-tutorial-jupyter
# Jupyter hub
http://jupyterhub.readthedocs.io/en/latest/index.html
#########
### data science
# websites
http://www.datasciencecentral.com/
# Data Science Specialization
http://datasciencespecialization.github.io/
https://github.com/DataScienceSpecialization/DataScienceSpecialization.github.io#contributing
# data science compared to other analytic disciplines
http://www.datasciencecentral.com/profiles/blogs/17-analytic-disciplines-compared
# Life Cycle of Data Science Projects
http://www.datasciencecentral.com/profiles/blogs/life-cycle-of-data-science-projects
# Becoming a Data Scientist – Curriculum via Metromap
http://nirvacana.com/thoughts/becoming-a-data-scientist/
# Data Wrangler
http://vis.stanford.edu/wrangler
# Rules for Scientific Data
http://www.ploscompbiol.org/article/info%3Adoi%2F10.1371%2Fjournal.pcbi.1003542
# FigShare repository: citable, shareable, and discoverable
http://figshare.com
# Zenodo repository: works with GitHub
http://zenodo.org
# MINE: mutual information NonParametric Data Exploration - R package to C++ functions - mostly bio
http://www.exploredata.net/
http://www.exploredata.net/Technical-information
http://www.eecs.harvard.edu/~michaelm/
http://www.mybiasedcoin.blogspot.com/
http://www.sabetilab.org/
http://intelligenttradingtech.blogspot.com/2012/01/there-was-lot-of-buzz-in-blogosphere-as.html
#########
### numerical methods dimension reduction
# Juha Karhunen - Independent Component Analysis
http://users.ics.aalto.fi/juha/
# Independent Component Analysis (ICA)
http://www.cis.hut.fi/Opinnot/T-61.6030/k2006/
http://cis.legacy.ics.tkk.fi/aapo/papers/IJCNN99_tutorialweb/
http://research.ics.aalto.fi/ica/icademo/
# David S. Matteson R package steadyICA Dimension Reduction for Multivariate Time Series
https://davidsmatteson.com/
http://ec2-52-90-141-138.compute-1.amazonaws.com:3838/
# Bryan W. Lewis at SciDB - package irlba for SVD and PCA
http://illposed.net/
http://bwlewis.github.io/GLM
http://bwlewis.github.io/cassini/
#########
### lectures and tutorials
# Learn How to Code - Where to start
http://gettinggeneticsdone.blogspot.com/2012/01/new-years-resolution-learn-how-to-code.html
# Tutorials
http://www.tutorialspoint.com/index.htm
http://www.tutorialspoint.com/shorttutorials/run-your-first-program-in-mingw-compiler
# Vivian Zhang
http://nycdatascience.com/
# Code School: Ruby, Java, and web technologies
https://www.codeschool.com/
#########
### languages and systems
# JSON is a replacement for XML, YAML is a superset of JSON
https://en.wikipedia.org/wiki/JSON
#########
### julia language for technical computing
http://julialang.org/
http://strata.oreilly.com/2012/10/matlab-r-julia-languages-for-data-analysis.html
# JuliaQuant Quantitative Finance in Julia
https://github.com/JuliaQuant
### Church probabilistic programming language
http://projects.csail.mit.edu/church/wiki/Church
# Probabilistic Models of Cognition by Noah D. Goodman
https://probmods.org/index.html
http://web.stanford.edu/~ngoodman/
# regex regular expression tutorial
http://www.coppelia.io/quick-start-regex-for-analysts-part-i/
http://www.coppelia.io/category/regex/
#########
### version control
# version control
http://stackoverflow.com/questions/406038/subversion-usages/406176#406176
# mercurial - modern, open source, distributed version control system
http://hginit.com/
# great review of Distributed Version Control Systems: GIT killer feature git-gisect
http://www.infoq.com/articles/dvcs-guide
# Source Tree version control
http://www.sourcetreeapp.com/
#########
### Git version control
### Git Home
http://git-scm.com/
http://en.wikipedia.org/wiki/Git_(software)
http://en.wikipedia.org/wiki/GitHub
### Git learn
# Git beginner guides
http://gitimmersion.com/
http://backlogtool.com/git-guide/en/