Stackoverflow Exception Entity Framework 3.1.3 when Seeding Large Dataset

entity-framework entity-framework-core

Question

When running the update-database command to my mssql server to seed a large dataset parsed from a CSV I am getting a stack overflow exception.

I seed the data from a number of legacy CSV files in OnModelCreating.

It is a large data set and contains around 9.5k lines of input which is split across a number of POCOs.

-verbose gives no further information on the issue other than it is trying to apply the migration.

I am using Visual Studio 2019 Community and EntityFrameworkCore 3.1.3.

Can I safely comment out some of the seeding (i.e. seed 50%) in the up() method of the migration?

How would I then seed the remaining data safely? (would EF on the OnModelBuilding method check the data on the next add-migration and generate the missing data? or would it assume it was seeded after checking the migrations table?

1
0
3/27/2020 4:06:35 PM

Accepted Answer

Your initialcreate.Up(MigrationBuilder) method is too big and is blowing up the JITter. The StackOverflow is occuring when the MSIL for the method is compiled to machine language code.

This is because there's just too much code of the form

  migrationBuilder.InsertData(
        table: "Addresses",
        columns: new[] { "Id", "Address1", "Address2", "Address3", "Country", "DateCreated", "IsArchived", "IsPrimaryAddress", "LastModifiedDate", "LastModifiedUser", "NickName", "Notes", "PostCode", "Region", "Town" },
        values: new object[,]
        {
            { 1, "669A George Street", null, null, "", new DateTime(2020, 3, 28, 14, 58, 51, 108, DateTimeKind.Utc).AddTicks(1514), null, true, new DateTime(2020, 3, 28, 14, 58, 51, 108, DateTimeKind.Utc).AddTicks(1545), null, null, null, "AB25 3XP", "", "Aberdeen" },
            { 12307, "7 Moredon Road", null, null, "", new DateTime(2020, 3, 28, 14, 58, 51, 198, DateTimeKind.Utc).AddTicks(8799), null, true, new DateTime(2020, 3, 28, 14, 58, 51, 198, DateTimeKind.Utc).AddTicks(8800), null, null, null, "SN25 3DQ ", "", "Swindon" },
            { 12306, "", null, null, null, new DateTime(2020, 3, 28, 14, 58, 51, 198, DateTimeKind.Utc).AddTicks(8735), null, false, new DateTime(2020, 3, 28, 14, 58, 51, 198, DateTimeKind.Utc).AddTicks(8736), null, null, null, null, null, null },
            { 12305, "67 HIGH STREET, ", null, null, "", new DateTime(2020, 3, 28, 14, 58, 51, 198, DateTimeKind.Utc).AddTicks(8690), null, true, new DateTime(2020, 3, 28, 14, 58, 51, 198, DateTimeKind.Utc).AddTicks(8690), null, null, null, "TQ95 5NU", "", "TOTNES" },
            { 12304, "", null, null, null, new DateTime(2020, 3, 28, 14, 58, 51, 198, DateTimeKind.Utc).AddTicks(8577), null, false, new DateTime(2020, 3, 28, 14, 58, 51, 198, DateTimeKind.Utc).AddTicks(8579), null, null, null, null, null, null },
            { 12303, "BOWS KITCHEN, 10 MARGATE PLACE,", null, null, "", new DateTime(2020, 3, 28, 14, 58, 51, 198, DateTimeKind.Utc).AddTicks(8530), null, true, new DateTime(2020, 3, 28, 14, 58, 51, 198, DateTimeKind.Utc).AddTicks(8531), null, null, null, "CT9 1EN", "", "MARGATE" },
            { 12302, "", null, null, null, new DateTime(2020, 3, 28, 14, 58, 51, 198, DateTimeKind.Utc).AddTicks(8464), null, false, new DateTime(2020, 3, 28, 14, 58, 51, 198, DateTimeKind.Utc).AddTicks(8465), null, null, null, null, null, null },
            { 12308, "", null, null, null, new DateTime(2020, 3, 28, 14, 58, 51, 198, DateTimeKind.Utc).AddTicks(8849), null, false, new DateTime(2020, 3, 28, 14, 58, 51, 198, DateTimeKind.Utc).AddTicks(8850), null, null, null, null, null, null },
            { 12301, "270 ST SAVIOURS RD", null, null, "", new DateTime(2020, 3, 28, 14, 58, 51, 198, DateTimeKind.Utc).AddTicks(8417), null, true, new DateTime(2020, 3, 28, 14, 58, 51, 198, DateTimeKind.Utc).AddTicks(8418), null, null, null, "LE5 4HF", "", "LEICESTER" },

in the method. To resolve this, the best method would be to read the data for each InserData from disk before each call. Or to embed the data in your source code as, say JSON strings and convert it to an object[,] at runtime.

1
3/28/2020 4:49:13 PM


Related Questions





Related

Licensed under: CC-BY-SA with attribution
Not affiliated with Stack Overflow
Licensed under: CC-BY-SA with attribution
Not affiliated with Stack Overflow