Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

No new data in database #14

Open
ggaukin opened this issue Apr 22, 2015 · 1 comment
Open

No new data in database #14

ggaukin opened this issue Apr 22, 2015 · 1 comment

Comments

@ggaukin
Copy link

ggaukin commented Apr 22, 2015

This is a strange issue and I have been working on it for over a week and cannot figure out the issue. This is a new build Ubuntu 14.04. The install.sh file went through without an issue but for some reason I am not getting any new data in the database. I received 100 logs and that is all. If I reboot it I receive 100 more additional logs. If I manually execute syslog-ng -Fevd it shows a multitude of data on screen. I see no issues in any of the log files. If I log into mysql and run select * from tables; I see that the start and end times of the syslog_data.syslogs_index_1 table are 10 seconds apart.

There is 1 exception I have found. If i execute livetail.pl I see everything that elsa is doing and all of that data is put into the database but only searchable from the archive. The moment I end the livetail the logs stop showing up in the database. I cannot figure where the disconnect is. Please assist in troubleshooting. Thank you.

@PVi1
Copy link

PVi1 commented Mar 10, 2017

The same problem here.

Fixed by commenting in fags(fow-control) and restating syslog-ng:
log { source(s_network); source(s_realtime); rewrite(r_host); rewrite(r_cisco_program); rewrite(r_snare); rewrite(r_from_pipes); rewrite(r_pipes); parser(p_db); rewrite(r_extracted_host); ###FILTER_UNPARSED###log { filter(f_unclassified); rewrite(r_unparsed); destination(d_unclassified); flags(final); }; log { destination(d_elsa); }; log { destination(d_debug); }; #flags(flow-control); };

Actually after few days tweaking syslog-ng, I see DB is indexing data but returns no results, only error:
`query: SELECT CONCAT(SUBSTR(type, 1, 4), "_", id) AS name, start AS start_int, FROM_UNIXTIME(start) AS start,
end AS end_int, FROM_UNIXTIME(end) AS end, type, last_id-first_id AS records, index_schema
FROM syslog.indexes WHERE type="temporary" OR (type="permanent" AND ISNULL(locked_by)) OR type="realtime" ORDER BY start
values:

  • ERROR [2017/03/10 10:37:34] /usr/local/elsa/web/lib/SyncMysql.pm (64) SyncMysql::query 26472 [undef]
    Query: SELECT CONCAT(SUBSTR(type, 1, 4), "_", id) AS name, start AS start_int, FROM_UNIXTIME(start) AS start,
    end AS end_int, FROM_UNIXTIME(end) AS end, type, last_id-first_id AS records, index_schema
    FROM syslog.indexes WHERE type="temporary" OR (type="permanent" AND ISNULL(locked_by)) OR type="realtime" ORDER BY start with values got error JSON text must be an object or array (but found number, string, true, false or null, use allow_nonref to allow this) at /usr/local/elsa/web/lib/Utils.pm line 264.
  • ERROR [2017/03/10 10:37:34] /usr/local/elsa/web/lib/Utils.pm (274) Utils::ANON 26472 [undef]
    No indexes, rv: 0
    `

Before I disabled archive, at east searching from archive (archive:1) worked.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants