Inital commit of the searchcode server 1.2.2 application which is lic…

…enced

under fair source usage for 5 users.

.gitignore

@@ -1,12 +1,21 @@
-*.class
+# Intellij
+.idea/
+*.iml
+*.iws
 
-# Mobile Tools for Java (J2ME)
-.mtj.tmp/
+# Mac
+.DS_Store
 
-# Package Files #
-*.jar
-*.war
-*.ear
+# Maven
+log/
+target/
 
-# virtual machine crash logs, see http://www.java.com/en/download/help/error_hotspot.xml
-hs_err_pid*
+# Custom
+repo/
+index/
+index_test/
+release/
+*.pyc
+*.tar.gz
+*.log
+*.log.*

LICENSE.txt

@@ -0,0 +1,44 @@
+Fair Source License, version 0.9
+
+Copyright (C) 2016 Ben Boyter
+
+Licensor: Boyter Online Services
+
+Software: searchcode server 1.2.2
+
+Use Limitation: 5 users
+
+License Grant. Licensor hereby grants to each recipient of the
+Software ("you") a non-exclusive, non-transferable, royalty-free and
+fully-paid-up license, under all of the Licensor’s copyright and
+patent rights, to use, copy, distribute, prepare derivative works of,
+publicly perform and display the Software, subject to the Use
+Limitation and the conditions set forth below.
+
+Use Limitation. The license granted above allows use by up to the
+number of users per entity set forth above (the "Use Limitation"). For
+determining the number of users, "you" includes all affiliates,
+meaning legal entities controlling, controlled by, or under common
+control with you. If you exceed the Use Limitation, your use is
+subject to payment of Licensor’s then-current list price for licenses.
+
+Conditions. Redistribution in source code or other forms must include
+a copy of this license document to be provided in a reasonable
+manner. Any redistribution of the Software is only allowed subject to
+this license.
+
+Trademarks. This license does not grant you any right in the
+trademarks, service marks, brand names or logos of Licensor.
+
+DISCLAIMER. THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OR
+CONDITION, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO WARRANTIES
+OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
+NONINFRINGEMENT. LICENSORS HEREBY DISCLAIM ALL LIABILITY, WHETHER IN
+AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
+CONNECTION WITH THE SOFTWARE.
+
+Termination. If you violate the terms of this license, your rights
+will terminate automatically and will not be reinstated without the
+prior written consent of Licensor. Any such termination will not
+affect the right of others who may have received copies of the
+Software from you.

README.txt

@@ -0,0 +1,134 @@
+To run searchcode server you will need to have any Windows, Linux or OSX machine with Java 8 installed.
+Testing and packaging was done using the below version.
+
+java version "1.8.0_65"
+Java(TM) SE Runtime Environment (build 1.8.0_65-b17)
+Java HotSpot(TM) 64-Bit Server VM (build 25.65-b01, mixed mode)
+
+Uncompress the file you have downloaded to a directory where you want to run searchcode server.
+This directory should have more disk space than the size of the repositories you want to index.
+
+Once unpacked assuming that java is in your path (check with the command java -version) you should be able to run
+searchcode with the following command for Linux/OSX/BSD
+
+./searchcode-server.sh
+
+or for Windows
+
+searchcode-server.bat
+
+After a few moments searchcode server should be ready to run. By default it will be running on port 8080.
+To connect to it enter the following in the browser,
+
+http://SERVER_IP:8080
+
+Be sure to replace SERVER_IP with the ip address if your server or localhost if running locally.
+If you see a page with a search bar then everything is fine.
+
+For further control you may want to edit the above files and include the java -Xmx arguement to specify the
+amount of RAM to use or any other java option you wish to pass in.
+
+To administer your searchcode server instance you need to click on the Admin link in the top right.
+Enter the default password Adm1n234 (change this via the properties file)
+to add git repositories. If you need help check the documentation page (link at the bottom
+of every page).
+
+Upgrades
+--------
+
+To upgrade your current instance of searchcode perform the following steps.
+
+* Stop your current instance of searchcode server
+* Make a backup copy of your current instances searchcode.properties and searchcode.sqlite files.
+* Uncompress the package to a new directory.
+* You can either
+    * Copy the uncompressed files over your current instance overwriting if prompted.
+    * Copy the directory dependancy-jars and all contents overwriting your current
+    * Copy the following files searchcode-1.2.2.jar searchcode-server.bar and searchcode-server.sh to your instance directory
+* Start your instance again
+
+It is also worth comparing your searchcode.properties file to the new one (or the documentation page) as there may be new configuration that you can use. Also be sure to check the settings page as there is likely to be new settings you can use.
+
+
+Change Log
+==========
+
+17 August 2016 - 1.2.2
+ - Add -w option to external git to avoid whitespace blame issue
+ - Resolve bug where some results were searchable but not browsable
+ - Resolve SVN cert prompt issue causing SVN to never checkout
+ - Fix potential performance issues with spell checker
+
+28 June 2016 - 1.2.1
+ - Resolve string exception bug when using local git on some files
+ - Resolve bug where empty username, language or repository caused indexing to throw exception
+ - Add additional information about resolving ulimit errors to documentation
+
+09 June 2016 - 1.2.0
+ - Add SVN repository support
+ - Resolve highlight issue where overlapping matching text caused issues
+ - Add search operators AND OR NOT and wildcard E.G. searc*
+ - Resolve bug where some files when clicked would be labeled as 404 while being searchable
+
+30 May 2016 - 1.1.3
+ - New property added to control depth of the file to index
+
+27 May 2016 - 1.1.2
+ - Additional resolve bug on RedHat/CentOS where file owner always resolves as "unknown"
+ - Additional memory improvements
+ - New properties added to control size of indexing queue size
+
+20 May 2016 - 1.1.1
+ - Resolve bug on RedHat/CentOS where file owner always resolves as "unknown"
+ - Adjust client side cache times to mitigate sleeping laptop cache issue
+ - Small performance tweaks
+
+13 May 2016 - 1.1.0
+ - Updated to allow indexing of named branches
+ - Additional memory improvements
+ - Logging of exceptions/warnings to rolling file
+ - Logging level support added via properties file
+ - Caching improvements
+ - Resolve issue where sometimes .git directory files would be indexed
+ - Improve delta calculation between revisions to improve indexing speed
+ - Fix bug where deleted repository indexing jobs would continue to run
+ - Fix bug where missing source could cause clicking through to result bug
+ - Additional file types added to file classifier
+ - Improved COCOMO calculation
+ - API Endpoint addition allowing adding, removing and list of repositories
+ - Increase indexing priority to ensure index refreshes happen faster
+ - Allow use of tilde ~ in repository names
+ - Improvements to index/clone when server unexpectedly stopped
+ - Paging added to the repository page
+ - Many additional bug fixes
+
+20 April 2016 - 1.0.3
+ - Improve memory performance across indexing and searching
+ - Add additional parameters allowing memory usage to be configured
+ - Resolve null pointer exceptions in various areas of the search pipeline
+ - Fix removal of repository logic
+
+14 April 2016 - 1.0.2
+ - Fix bug where unable to add new repository
+
+13 April 2016 - 1.0.1
+ - Updated indexer code to improve performance
+ - Fix MD5 file calculation bug
+ - Add owner and user filter into index
+ - Add Match Lines, Max Line Depth, Minified Length to settings page
+ - Change links to files away from document id to path
+ - Added splitting of terms where possible such as ArrayList into Array List
+ - Improved match relevance for searches
+ - Improved calculation of COCOMO to be more accurate
+ - Fixed overloaded issues when server under high load
+ - Updated language identifiation to pick up languages, Varnish Configuration, Razor Template, Jade Template, ReStructuredText, Razor Template, Handlebars Template
+ - Performance improvements for all pages using smart caching of reused assets as found through load tests
+
+31 March 2016 - 1.0.0
+ - Updated look and feel with new fancy single page application design
+ - Improved performance of search and match 3-20x 
+ - Additional configuration of properties file allowing reverting to pure HTML page if required
+ - Various bug fixes
+
+8 March 2016 - 0.9.9
+ - Inital release

assets/docker

@@ -0,0 +1,14 @@
+FROM java:openjdk-8-jre
+
+RUN apt-get update && apt-get install -y \
+    curl \
+    tar
+
+RUN curl -o searchcode-server-community.tar.gz https://searchcode.com/static/searchcode-server-community.tar.gz \
+    && tar xvfz searchcode-server-community.tar.gz
+
+WORKDIR /searchcode-server-community/release
+
+EXPOSE 8080
+
+CMD ["./searchcode-server.sh"]

assets/integration_test/fuzztest.py

@@ -0,0 +1,87 @@
+#!/usr/local/bin/python
+# coding: utf-8
+
+import unittest
+import urllib2
+import urllib
+import json
+import sys
+import re
+import os
+import random
+import string
+
+'''Throws all sorts of queries at the HTML endpoint to see if we get any non 200
+responses as well as a few other checks. Useful for shaking out performance issues as
+well as identifying anything that has suddenly broken.'''
+class TestIntegration(unittest.TestCase):
+    def getData(self, url):
+        data = urllib2.urlopen(url)
+        self.assertEqual(200, data.getcode())
+        data = data.read()
+        return data
+
+    def getRandomCharacters(self, count):
+        return ''.join(random.choice(string.printable) for i in xrange(count))
+
+    def getRandomLetters(self, count):
+        return ''.join(random.choice(string.letters) for i in xrange(count))
+
+
+    def testFuzzSearchLetters(self):
+        for x in xrange(1000):
+            one = urllib.urlencode({'q': self.getRandomLetters(10)})
+            two = urllib.urlencode({'repo': self.getRandomLetters(10)})
+
+            url = "http://%s/html/?%s&%s" % (host, one, two)
+            data = self.getData(url)
+            self.assertTrue('refine search' in data)
+
+    def testFuzzSearchLettersTilde(self):
+        for x in xrange(1000):
+            one = urllib.urlencode({'q': self.getRandomLetters(10)})
+            two = urllib.urlencode({'repo': '~' + self.getRandomLetters(10)})
+
+            url = "http://%s/html/?%s&%s" % (host, one, two)
+            data = self.getData(url)
+            self.assertTrue('refine search' in data)
+
+    def testFuzzSearchLettersTildeSlash(self):
+        for x in xrange(1000):
+            one = urllib.urlencode({'q': self.getRandomLetters(10)})
+            two = urllib.urlencode({'repo': '~' + self.getRandomLetters(10) + '/' + self.getRandomLetters(10) })
+
+            url = "http://%s/html/?%s&%s" % (host, one, two)
+            data = self.getData(url)
+            self.assertTrue('refine search' in data)
+
+    def testFuzzSearchCharacters(self):
+        for x in xrange(1000):
+            one = urllib.urlencode({'q': self.getRandomCharacters(10)})
+            two = urllib.urlencode({'repo': self.getRandomCharacters(10)})
+
+            url = "http://%s/html/?%s&%s" % (host, one, two)
+            data = self.getData(url)
+            self.assertTrue('refine search' in data)
+
+    def testFuzzSearchCharactersTilde(self):
+        for x in xrange(1000):
+            one = urllib.urlencode({'q': self.getRandomCharacters(10)})
+            two = urllib.urlencode({'repo': '~' + self.getRandomCharacters(10)})
+
+            url = "http://%s/html/?%s&%s" % (host, one, two)
+            data = self.getData(url)
+            self.assertTrue('refine search' in data)
+
+    def testFuzzSearchCharactersTildeSlash(self):
+        for x in xrange(1000):
+            one = urllib.urlencode({'q': self.getRandomCharacters(10)})
+            two = urllib.urlencode({'repo': '~' + self.getRandomCharacters(10)  + '/' + self.getRandomLetters(10) })
+
+            url = "http://%s/html/?%s&%s" % (host, one, two)
+            data = self.getData(url)
+            self.assertTrue('refine search' in data)
+
+if __name__ == "__main__":
+    host = "localhost:8080"
+    unittest.main()

assets/integration_test/signed_test.py

@@ -0,0 +1,47 @@
+from hashlib import sha1
+from hmac import new as hmac
+import urllib2
+import json
+import urllib
+
+'''Simple usage of the signed key API endpoints.'''
+
+
+publickey = "APIK-htEIV1L1GrOQhYRovctl2fkxdPf"
+privatekey = "IBrB9m9bbxBaoK86D8XcS7jtaA52rcok"
+
+for x in range(0, 10):
+    reponame = "myrepo%s" % (x) 
+    repourl = "/Users/boyter/test2"
+    repotype = "git"
+    repousername = ""
+    repopassword = ""
+    reposource = ""
+    repobranch = "master"
+
+    message = "pub=%s&reponame=%s&repourl=%s&repotype=%s&repousername=%s&repopassword=%s&reposource=%s&repobranch=%s" % (
+            urllib.quote_plus(publickey),
+            urllib.quote_plus(reponame),
+            urllib.quote_plus(repourl),
+            urllib.quote_plus(repotype),
+            urllib.quote_plus(repousername),
+            urllib.quote_plus(repopassword),
+            urllib.quote_plus(reposource),
+            urllib.quote_plus(repobranch)
+        )
+
+    message = "pub=%s&reponame=%s" % (
+            urllib.quote_plus(publickey),
+            urllib.quote_plus(reponame),
+        )
+
+    sig = hmac(privatekey, message, sha1).hexdigest()
+
+    url = "http://localhost:8080/api/repo/add/?sig=%s&%s" % (urllib.quote_plus(sig), message)
+    url = "http://localhost:8080/api/repo/delete/?sig=%s&%s" % (urllib.quote_plus(sig), message)
+
+    data = urllib2.urlopen(url)
+    data = data.read()
+
+    data = json.loads(data)
+    print x, data['sucessful'], data['message']