Implement elastic search issue indexer interface #6150

lunny · 2019-02-21T14:22:25Z

No description provided.

lunny · 2019-04-02T09:39:38Z

It's ready for review now.

modules/indexer/issues/elastic_search.go

codecov-io · 2019-04-02T10:00:38Z

Codecov Report

❗ No coverage uploaded for pull request base (master@46d3d10). Click here to learn what that means.
The diff coverage is 7.42%.

@@            Coverage Diff            @@
##             master    #6150   +/-   ##
=========================================
  Coverage          ?   40.14%           
=========================================
  Files             ?      404           
  Lines             ?    54263           
  Branches          ?        0           
=========================================
  Hits              ?    21786           
  Misses            ?    29461           
  Partials          ?     3016

Impacted Files	Coverage Δ
modules/indexer/issues/elastic_search.go	`0% <0%> (ø)`
modules/setting/indexer.go	`100% <100%> (ø)`
modules/indexer/issues/indexer.go	`61.9% <50%> (ø)`

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 46d3d10...6886071. Read the comment docs.

zeripath

Hi lunny!

Looks good overall ~~however I think you need to do a general search and replace of elestic for elastic.~~ Done!

~~Your choice of a skip of 2 for the elastic logger seems unusual. 1 is usually correct as it points to where the Printf is called, unless elastic wraps the Printf itself in which case 2 is appropriate.~~ I've just doublechecked the source. You're right!

You still need to add config stubs to modules/setting/log.go. Basically you need a setting ELASTIC which configures which sublogging modes that would use. You could make this generic and name it INDEXER? Then you could use the same named logger for different types of indexer?

modules/indexer/issues/elastic_search.go

lafriks · 2019-04-04T07:57:22Z

modules/indexer/issues/elastic_search.go

+
+	if !exists {
+		mapping := `{
+			"settings":{


Would be nice if this could be moved out to embedded file so that it could be customized using custom directory

Maybe a config item on app.ini to point to a mapping file which default are empty is better?

that could also be an option

.drone.yml

integrations/mysql.ini.tmpl

Cellebyte · 2019-05-08T08:42:26Z

@lunny do you know how long this will take to be implemented?
Devs are complaining on the bad codesearch in gitea. I know that DBs and Bleave are not the best search providers.

lunny · 2019-05-08T13:02:34Z

These PRs should my next work since migration PR merged. But I can't give you some date.

Cellebyte · 2019-05-08T13:17:16Z

Thanks so it will definitly be in the next version 1.9.x?

techknowlogick · 2019-07-07T04:23:01Z

conflicts

…t for elastic search

emonty · 2019-11-06T07:50:44Z

@lunny I got this: https://zuul.opendev.org/t/openstack/build/ddba351742a84e66947928231e6a02ff/log/gitea99.opendev.org/docker/giteadocker_gitea-web_1.txt#5319 which maybe looks like the code is trying to create the elasticsearch index twice? there's still something else wrong with that patch, so I'm not 100% sure yet

emonty · 2019-11-06T08:27:30Z

@lunny nevermind. that's a log line about repo indexer. I think I found another issue in my test job - I'll let you know how the latest version goes

lunny · 2019-12-19T09:41:46Z

Replaced by #9428

lunny added the type/enhancement An improvement of existing functionality label Feb 21, 2019

lafriks added this to the 1.9.0 milestone Feb 21, 2019

lunny force-pushed the lunny/issue_indexer_es branch 2 times, most recently from bef5278 to 316de4c Compare April 2, 2019 09:37

lunny changed the title ~~WIP: implement elastic search issue indexer interface~~ Implement elastic search issue indexer interface Apr 2, 2019

GiteaBot added the lgtm/need 2 This PR needs two approvals by maintainers to be considered for merging. label Apr 2, 2019

apricote suggested changes Apr 2, 2019

View reviewed changes

modules/indexer/issues/elastic_search.go Outdated Show resolved Hide resolved

zeripath reviewed Apr 2, 2019

View reviewed changes

modules/indexer/issues/elastic_search.go Show resolved Hide resolved

lunny force-pushed the lunny/issue_indexer_es branch from 316de4c to 44637ed Compare April 3, 2019 08:38

zeripath approved these changes Apr 3, 2019

View reviewed changes

GiteaBot added lgtm/need 1 This PR needs approval from one additional maintainer to be merged. and removed lgtm/need 2 This PR needs two approvals by maintainers to be considered for merging. labels Apr 3, 2019

lafriks reviewed Apr 4, 2019

View reviewed changes

lunny force-pushed the lunny/issue_indexer_es branch from 44637ed to 8f20faf Compare April 4, 2019 14:53

apricote reviewed Apr 4, 2019

View reviewed changes

.drone.yml Outdated Show resolved Hide resolved

apricote reviewed Apr 5, 2019

View reviewed changes

.drone.yml Outdated Show resolved Hide resolved

techknowlogick reviewed Apr 5, 2019

View reviewed changes

integrations/mysql.ini.tmpl Outdated Show resolved Hide resolved

lunny force-pushed the lunny/issue_indexer_es branch from 49ac4b8 to 1bf75b6 Compare April 12, 2019 08:13

lunny mentioned this pull request Apr 16, 2019

Support better search engines for indexing (elasticsearch, mysql/pgsql/sqlite fulltext or something else) #6648

Closed

1 task

lunny force-pushed the lunny/issue_indexer_es branch from 1bf75b6 to ab901c9 Compare April 16, 2019 14:43

lunny force-pushed the lunny/issue_indexer_es branch from ab901c9 to ceec710 Compare May 8, 2019 13:30

appleboy approved these changes May 31, 2019

View reviewed changes

GiteaBot added lgtm/done This PR has enough approvals to get merged. There are no important open reservations anymore. and removed lgtm/need 1 This PR needs approval from one additional maintainer to be merged. labels May 31, 2019

lunny force-pushed the lunny/issue_indexer_es branch from ceec710 to d9699ef Compare May 31, 2019 14:40

techknowlogick added this to the 1.10.0 milestone Jun 4, 2019

lunny force-pushed the lunny/issue_indexer_es branch from 63e9a4f to cdce7f9 Compare June 10, 2019 11:15

lunny force-pushed the lunny/issue_indexer_es branch from cdce7f9 to e81618c Compare July 7, 2019 07:39

lunny added 10 commits July 9, 2019 14:19

implement elastic search issue indexer interface

314d5ea

added vendors

f12b32b

switch to go mod

3431943

fix vet

ed6a3a2

finish single elastic search node index and search

022c0ab

improve the logger

a785e69

batch index and delete

c800ca3

add elastic search issue indexer custom mapping file support; add tes…

7a5581d

…t for elastic search

fix elastic search test

e5bead4

fix integration mysql.ini.tmpl

6886071

lunny force-pushed the lunny/issue_indexer_es branch from e81618c to 6886071 Compare July 9, 2019 06:20

techknowlogick modified the milestones: 1.10.0, 1.11.0 Sep 3, 2019

lunny added the status/blocked This PR cannot be merged yet, i.e. because it depends on another unmerged PR label Sep 18, 2019

update vendor

40b7f55

lunny force-pushed the lunny/issue_indexer_es branch from 8559553 to 40b7f55 Compare November 6, 2019 03:37

lunny modified the milestones: 1.11.0, 1.12.0 Dec 13, 2019

lunny mentioned this pull request Dec 19, 2019

Issue search support elasticsearch #9428

Merged

lunny closed this Dec 19, 2019

lunny deleted the lunny/issue_indexer_es branch December 19, 2019 09:40

lunny removed this from the 1.12.0 milestone Dec 19, 2019

go-gitea locked and limited conversation to collaborators Nov 24, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Implement elastic search issue indexer interface #6150

Implement elastic search issue indexer interface #6150

lunny commented Feb 21, 2019

lunny commented Apr 2, 2019

codecov-io commented Apr 2, 2019 •

edited

Loading

zeripath left a comment •

edited

Loading

lafriks Apr 4, 2019

lunny Apr 4, 2019 •

edited

Loading

lafriks Apr 5, 2019

Cellebyte commented May 8, 2019

lunny commented May 8, 2019

Cellebyte commented May 8, 2019

techknowlogick commented Jul 7, 2019

emonty commented Nov 6, 2019

emonty commented Nov 6, 2019

lunny commented Dec 19, 2019

Implement elastic search issue indexer interface #6150

Implement elastic search issue indexer interface #6150

Conversation

lunny commented Feb 21, 2019

lunny commented Apr 2, 2019

codecov-io commented Apr 2, 2019 • edited Loading

Codecov Report

zeripath left a comment • edited Loading

Choose a reason for hiding this comment

lafriks Apr 4, 2019

Choose a reason for hiding this comment

lunny Apr 4, 2019 • edited Loading

Choose a reason for hiding this comment

lafriks Apr 5, 2019

Choose a reason for hiding this comment

Cellebyte commented May 8, 2019

lunny commented May 8, 2019

Cellebyte commented May 8, 2019

techknowlogick commented Jul 7, 2019

emonty commented Nov 6, 2019

emonty commented Nov 6, 2019

lunny commented Dec 19, 2019

codecov-io commented Apr 2, 2019 •

edited

Loading

zeripath left a comment •

edited

Loading

lunny Apr 4, 2019 •

edited

Loading