I am currently trying to sync two SQLite databases with EF Core. To do this I have a lot of data manipulation that needs to be tracked. I can't do this with EF Core because of memory issues (about 5 million records for each database).
So I was thinking about trying to save sorted and changed data into temp tables for the processing. Then pull the data out to save all back to the databases once processing is complete. Another option is to save to XML list on disk.
Any help, advice, best practice would be great.
You could use stored procedure, or just Raw SQL Queries
Entity Framework Core allows you to drop down to raw SQL queries when working with a relational database. This can be useful if the query you want to perform can't be expressed using LINQ, or if using a LINQ query is resulting in inefficient SQL being sent to the database.
var blogs = context.Blogs .FromSql("SELECT * FROM dbo.Blogs") .ToList();
However, do note there are many limitations