evtx fields that need to be added to index template

[mmguero]

Some of the EVTX fields coming from the evtx utility need to be normalized. Here's a sanitized version of the output from logstash. The files used are here.

 [evtx.Event.EventData.Action] of type [long] Preview of field's value: 
 [evtx.Event.EventData.Action] of type [long] Preview of field's value: '%%16389'
 [evtx.Event.EventData.Action] of type [long] Preview of field's value: '%%16390'
 [evtx.Event.EventData.Action] of type [long] Preview of field's value: '%%16391'
 [evtx.Event.EventData.Category] of type [integer] Preview of field's value: 'Inventory'
 [evtx.Event.EventData.Command] of type [long] Preview of field's value: '22.012.0117.0003\"'
 [evtx.Event.EventData.Command] of type [long] Preview of field's value: 'OneDrive.exe\" /background'
 [evtx.Event.EventData.Command] of type [long] Preview of field's value: 'OneDriveSetup.exe /thfirstsetup'
 [evtx.Event.EventData.Command] of type [long] Preview of field's value: 'OneDriveSetup.exe\"'
 [evtx.Event.EventData.Command] of type [long] Preview of field's value: 'SecurityHealthSystray.exe'
 [evtx.Event.EventData.Command] of type [long] Preview of field's value: 'chrmstp.exe\" --configure-user-settings --verbose-logging --system-level --channel=stable'
 [evtx.Event.EventData.Command] of type [long] Preview of field's value: 'ie4uinit.exe -UserConfig'
 [evtx.Event.EventData.Command] of type [long] Preview of field's value: 'installer.exe\" /repair'
 [evtx.Event.EventData.Command] of type [long] Preview of field's value: 'powershell Add-AppxPackage -RegisterByFamilyName -MainPackage Microsoft.DesktopAppInstaller_8wekyb3d8bbwe'
 [evtx.Event.EventData.Command] of type [long] Preview of field's value: 'powershell Add-AppxPackage -RegisterByFamilyName -MainPackage Microsoft.WindowsTerminal_8wekyb3d8bbwe'
 [evtx.Event.EventData.Command] of type [long] Preview of field's value: 'setup.exe\" --configure-user-settings --verbose-logging --system-level --msedge --channel=stable'
 [evtx.Event.EventData.Command] of type [long] Preview of field's value: 'setup.exe\" --msedgewebview --delete-old-versions --system-level --verbose-logging --on-logon'
 [evtx.Event.EventData.Command] of type [long] Preview of field's value: 'unregmp2.exe /FirstLogon'
 [evtx.Event.EventData.Command] of type [long] Preview of field's value: 'vmtoolsd.exe\" -n vmusr'
 [evtx.Event.EventData.Context] of type [long] Preview of field's value: 'onecore\\ds\\security\\gina\\profile\\profext\\appcontainer.cpp Line:1862 Usermode Font Driver Host microsoft.windows.fontdrvhost'
 [evtx.Event.EventData.Device] of type [long] Preview of field's value: '\\Device\\Harddisk0\\DR0'
 [evtx.Event.EventData.Device] of type [long] Preview of field's value: '\\Device\\HarddiskVolume1'
 [evtx.Event.EventData.Device] of type [long] Preview of field's value: '\\Device\\HarddiskVolume2'
 [evtx.Event.EventData.Device] of type [long] Preview of field's value: '\\Device\\HarddiskVolume3'
 [evtx.Event.EventData.Device] of type [long] Preview of field's value: '\\Device\\HarddiskVolume4'
 [evtx.Event.EventData.Error] of type [long] Preview of field's value: '0x1'
 [evtx.Event.EventData.Error] of type [long] Preview of field's value: '0x1f'
 [evtx.Event.EventData.Error] of type [long] Preview of field's value: '0x2'
 [evtx.Event.EventData.Error] of type [long] Preview of field's value: '0xc000014f'
 [evtx.Event.EventData.Error] of type [long] Preview of field's value: 'Windows cannot install package Microsoft.VCLibs.140.00_14.0.30704.0_x64__8wekyb3d8bbwe because it has version 14.0.30704.0. A higher version 14.0.32530.0 of this package is already installed.'
 [evtx.Event.EventData.Flags] of type [integer] Preview of field's value: '0x10'
 [evtx.Event.EventData.Flags] of type [integer] Preview of field's value: '0x103'
 [evtx.Event.EventData.Flags] of type [integer] Preview of field's value: '0x112'
 [evtx.Event.EventData.Flags] of type [integer] Preview of field's value: '0x2'
 [evtx.Event.EventData.Flags] of type [integer] Preview of field's value: '0x20'
 [evtx.Event.EventData.Flags] of type [integer] Preview of field's value: '0x20000'
 [evtx.Event.EventData.Flags] of type [integer] Preview of field's value: '0x20040'
 [evtx.Event.EventData.Flags] of type [integer] Preview of field's value: '0x40'
 [evtx.Event.EventData.Flags] of type [integer] Preview of field's value: '0x5'
 [evtx.Event.EventData.Flags] of type [integer] Preview of field's value: '0x9'
 [evtx.Event.EventData.Flags] of type [integer] Preview of field's value: '1126037345796100'
 [evtx.Event.EventData.Flags] of type [integer] Preview of field's value: '2226412716160'
 [evtx.Event.EventData.Flags] of type [integer] Preview of field's value: '2449474112'
 [evtx.Event.EventData.Flags] of type [integer] Preview of field's value: '4535619747840'
 [evtx.Event.EventData.Flags] of type [integer] Preview of field's value: '4535619747848'
 [evtx.Event.EventData.Flags] of type [integer] Preview of field's value: '4535619747872'
 [evtx.Event.EventData.Flags] of type [integer] Preview of field's value: '7206885441476231200'
 [evtx.Event.EventData.Function] of type [long] Preview of field's value: 'CreateLicenseManager'
 [evtx.Event.EventData.Function] of type [long] Preview of field's value: 'DriverEntry'
 [evtx.Event.EventData.Function] of type [long] Preview of field's value: 'NetworkStatusMonitor::Initialize'
 [evtx.Event.EventData.Function] of type [long] Preview of field's value: 'OneStoreApplicationLicenseManager::GetInstalledLicensesWithMinQuality'
 [evtx.Event.EventData.Function] of type [long] Preview of field's value: 'RootMachine::RootMachine'
 [evtx.Event.EventData.Function] of type [long] Preview of field's value: 'ServiceInitializeImpl'
 [evtx.Event.EventData.Function] of type [long] Preview of field's value: 'StoredIdentity::StoredIdentity'
 [evtx.Event.EventData.Function] of type [long] Preview of field's value: 'WnfEventHandlerForDeviceIdChangeImpl'
 [evtx.Event.EventData.HRESULT] of type [long] Preview of field's value: '0x80070057'
 [evtx.Event.EventData.IsMachine] of type [boolean] Preview of field's value: '1'
 [evtx.Event.EventData.LogonType] of type [integer] Preview of field's value: 'Regular'
 [evtx.Event.EventData.Reason] of type [long] Preview of field's value: 'ExternalObjects'
 [evtx.Event.EventData.Reason] of type [long] Preview of field's value: 'None'
 [evtx.Event.EventData.Reason] of type [long] Preview of field's value: 'S-1-5-21-2533829718-189860685-2477588761-500'
 [evtx.Event.EventData.ReturnCode] of type [long] Preview of field's value: '0x80090016'
 [evtx.Event.EventData.SessionId] of type [long] Preview of field's value: '1a75463c-9592-4179-9503-aada9bd3b9ed'
 [evtx.Event.EventData.UserId] of type [long] Preview of field's value: 'S-1-5-19'
 [evtx.Event.EventData.UserId] of type [long] Preview of field's value: 'S-1-5-21-2533829718-189860685-2477588761-500'
 [evtx.Event.EventData.errorCode] of type [long] Preview of field's value: '0x800706be'
 [evtx.Event.EventData.errorCode] of type [long] Preview of field's value: '0x8024200b'
 [evtx.Event.EventData.value] of type [float] Preview of field's value: 'CDS'
 [evtx.Event.EventData.value] of type [float] Preview of field's value: 'Graveyard'
 [evtx.Event.EventData.value] of type [float] Preview of field's value: 'Start.Suggestions'
 [evtx.Event.EventData.value] of type [float] Preview of field's value: 'Start.TileGrid'
 [evtx.Event.EventData.value] of type [long] Preview of field's value: 'CDS'
 [evtx.Event.EventData.value] of type [long] Preview of field's value: 'Graveyard'
 [evtx.Event.EventData.value] of type [long] Preview of field's value: 'Start.Suggestions'
 [evtx.Event.EventData.value] of type [long] Preview of field's value: 'Start.TileGrid'

[mmguero]

To debug:

$ ./scripts/logs -s logstash | grep --line-buffered "failed to parse"  | sed "s/.*failed to parse field //" | sed 's/of type.*value: /: /' | sed 's/, "caused_by.*//'

[mmguero]

I think we got it