How to map non-public Collection properties with Entity Framework Code First?

I am trying to map a collection of entities with a non-public property, so as to better encapsulate my domain, since the entity that contains the collection needs to enforce certain rules on the collection, e.g.

  public class Foo
  {
      public AddBar(Bar bar)
      {
          if (BarsInternal.Count > 4)
          {
              throw new Exception("Too many Bars!");
          }
          BarsInternal.Add(bar);
      }           

      public IEnumerable<Bar> Bars 
      { 
          get { foreach (var bar in BarsInternal) { yield return bar; } }
      }

      internal virtual IList<Bar> BarsInternal { get; set; }

  }

For non-collection properties, I have in the past been able to do this by making the property internal, and simply mapping the property as normal (applying InternalsVisibleTo when entity framework was not in the same assembly as the domain model) using a mapping class.

However, for collections I cannot seem to get this to work. If I mark the property public, and use the following mapping:

public class FooMap : EntityTypeConfiguration<Foo>
{
    public FooMap()
    {
        this.HasMany(x => x.BarsInternal);
    }
}

Then, everything works.

However, if I change the Bars property to be internal, then I get the following odd behavior:

• The database tables are created as expected (using code first migrations), and the foreign keys and everything lines up, so apparently entity framework is reading the internal property in some fashion
• Code First Migrations using context.Foos.AddOrUpdate(fooInstance) does fully persist the Bars collection in the database, as expected.
• But, when retrieving a Foo from the context, the Bars collection is always empty.

How can I map a non-public collection property correctly in Entity Framework Code First?