When using SQL schema should be computed from the query, not the logs

[juan-carvajal]

Example schema:

* Schema for logstashlogs

1. @timestamp - Utf8

2. @version - Utf8

3. error - Utf8

4. event_original - Utf8

5. host_name - Utf8

6. latency - Int64

7. level - Utf8

8. log_file_path - Utf8

9. method - Utf8

10. mongoURI - Utf8

11. msg - Utf8

12. origin_host - Utf8

13. origin_ip - Utf8

14. origin_proxy - Utf8

15. p_metadata - Utf8

16. p_tags - Utf8

17. p_timestamp - {"Timestamp":["Millisecond",null]}

18. panic - Utf8

19. path - Utf8

20. referer - Utf8

21. service - Utf8

22. source_file - Utf8

23. source_function - Utf8

24. source_line - Int64

25. span_id - Utf8

26. stack_CreatedBy_File - {"List":{"name":"item","data_type":"Utf8","nullable":true,"dict_id":0,"dict_is_ordered":false,"metadata":{}}}

27. stack_CreatedBy_Func - {"List":{"name":"item","data_type":"Utf8","nullable":true,"dict_id":0,"dict_is_ordered":false,"metadata":{}}}

28. stack_CreatedBy_Line - {"List":{"name":"item","data_type":"Int64","nullable":true,"dict_id":0,"dict_is_ordered":false,"metadata":{}}}

29. stack_FramesElided - {"List":{"name":"item","data_type":"Boolean","nullable":true,"dict_id":0,"dict_is_ordered":false,"metadata":{}}}

30. stack_ID - {"List":{"name":"item","data_type":"Int64","nullable":true,"dict_id":0,"dict_is_ordered":false,"metadata":{}}}

31. stack_LockedToThread - {"List":{"name":"item","data_type":"Boolean","nullable":true,"dict_id":0,"dict_is_ordered":false,"metadata":{}}}

32. stack_Stack_File - {"List":{"name":"item","data_type":{"List":{"name":"item","data_type":"Utf8","nullable":true,"dict_id":0,"dict_is_ordered":false,"metadata":{}}},"nullable":true,"dict_id":0,"dict_is_ordered":false,"metadata":{}}}

33. stack_Stack_Func - {"List":{"name":"item","data_type":{"List":{"name":"item","data_type":"Utf8","nullable":true,"dict_id":0,"dict_is_ordered":false,"metadata":{}}},"nullable":true,"dict_id":0,"dict_is_ordered":false,"metadata":{}}}

34. stack_Stack_Line - {"List":{"name":"item","data_type":{"List":{"name":"item","data_type":"Int64","nullable":true,"dict_id":0,"dict_is_ordered":false,"metadata":{}}},"nullable":true,"dict_id":0,"dict_is_ordered":false,"metadata":{}}}

35. stack_State - {"List":{"name":"item","data_type":"Utf8","nullable":true,"dict_id":0,"dict_is_ordered":false,"metadata":{}}}

36. stack_Wait - {"List":{"name":"item","data_type":"Int64","nullable":true,"dict_id":0,"dict_is_ordered":false,"metadata":{}}}

37. status - Int64

38. status_code - Int64

39. target - Utf8

40. time - Utf8

41. trace_id - Utf8

42. type - Utf8

43. user-agent - Utf8 

Example query:
SELECT avg(latency) as lat,path,service,status,method from logstashlogs group by path,service,status,method

Resulting table:

Where is column lat?

This defeats the entire purpose of using custom SQL.

Btw, thanks for the software, just trying to help improve it.

[nitisht]

Can you be a little elaborate here @juan-carvajal - what is the issue here?

[nikhilsinhaparseable]

Query API gives proper response as expected, attached screenshot for reference.
@balaji-jr please check, query fired from the console and the response also looks good, only the table needs to be corrected.

[balaji-jr]

Hey @juan-carvajal,
This problem has been resolved and will be included in the upcoming release.
parseablehq/console#240.