Have you looked into Liquid Neural Networks? These networks vary algorithm parameters over time, enhancing the ability to analyze time series data.
The MIT researchers coded a neural network to emulate how C. elegans neurons activate and communicate with each other via electrical impulses. Parameters change over time based on the results of a nested set of differential equations. The initial research has been performed on computer vision tasks, but it seems this approach could be adopted for streaming tabular data. What do you think?
https://news.mit.edu/2021/machine-learning-adapts-0128