Multiline parser for python log format using fluent-bit. getting single line log entry on elastic while i want it to be a single block
sample log entry –
2024-06-14 06:51:58.906 | INFO | main:main:30 – All indicators and predictions:
[
{
“datetime”:1718308800000,
“key1”:66701.7292561551,
“key2”:67014.6836462447,
“key3”:66529.3890236763,
“key4”:66880.5703857987,
“key5”:1132.28416,
“key6”:744.0276710022,
“key7”:0,
“key8”:0.0585026458,
“key9”:0,
“key10″:””,
“key11”:null
}
]
2024-06-14 06:51:58.907 | INFO | main:main:31 – datetime key1 key2 key3 key4 key5 … key6 key7 key8 key9 key10 key11
994 2024-06-13 20:00:00 66701.729256 67014.683646 66529.389024 66880.570386 1132.28416 … 744.027671 0 0.058503 0 NaN
995 2024-06-13 22:00:00 66880.569752 66991.561215 66630.729675 66808.863462 788.89703 … 680.327224 0 0.036088 0 NaN
996 2024-06-14 00:00:00 66808.863251 66929.160405 66530.243225 66807.067821 1427.33212 … 660.197075 0 -0.004118 0 NaN
997 2024-06-14 02:00:00 66807.067750 66994.600135 66627.601075 66765.022607 1277.60587 … 648.589025 0 0.036305 0 NaN
998 2024-06-14 04:00:00 66765.022583 67145.746712 66681.040358 66956.667536 1253.85572 … 640.019675 0 0.151628 0 NaN