Ticket Q523295
Visible to All Users

Core - Improve XPObjectSpace performance when processing large amounts of data

created 11 years ago

Hello,

I have a performace problem related to the "objectsToSave"-ArrayList contained in the class XPObjectSpace.

I have a Session which is connected to an XPObjectSpace. I am creating lots of domainobjects within this Session (30000).
Each time after I have created 30 domainobjects I am calling Session.CommitTransaction() or UnitOfWork.CommitChanges().
I know that it may not be correct to do this. I think that I should call XPObjectSpace.CommitChanges() here.

The main problem is that each time the method SetPropertyValue is called on a domainobject then the method XPObjectSpace.SetModified will be called.

protected override void SetModified(Object obj, ObjectChangedEventArgs args) {
CheckIsDisposed();
if(obj != null) {
XPClassInfo objClassInfo = FindObjectXPClassInfo(obj);
if(objClassInfo != null && objClassInfo.IsPersistent) {
if(!objectsToSave.Contains(obj)) {
objectsToSave.Add(obj);
}

}

Because of the fact that "objectsToSave" is a ArrayList the method call objectsToSave.Contains(obj) will lead to an iteration over all domainobjects contained in this ArrayList (with lots of equals-checks). This will have a significant performance impact if you have a lot of uncommitted domainobjects in the XPObjectSpace. Wouldn't it be better to implement "objectsToSave" with a HashSet in order increase the perfomance in general.

Another minor problem:
The main problem occurs in the described scenario only because a call to Session.CommitTransaction() (or UnitOfWork.CommitChanges()) will not clear the "objectsToSave"-ArrayList in the according XPObjectSpace. This will only be done if XPObjectSpace.CommitChanges()
is called directly. Maybe the XPObjectSpace could track commits of the underlying Session and clear the "objectsToSave"-ArrayList. But this is not the very important here. I just wanted to mention it. I can solve this simply by calling XPObjectSpace.CommitChanges().

The main problem will arise if I want to create many domainobjects and want to commit them in a single database-transaction.
You can reproduce both problems with the attached sample project. Call the action "CreateTestData" and profile how must time is lost in method XPObjectSpace.SetModified().

Thanks

Frank

Answers approved by DevExpress Support

created 11 years ago (modified 11 years ago)

We have implemented the functionality described in this ticket. It will be included in our next update(s).

Please check back and leave a comment to this response to let us know whether or not this solution addresses your concerns.

Additional information:

The XPObjectSpace class was improved to speed up CRUD manipulations with large amount of data, e.g. for data import scenarios. In particular, creation and update operations are now ~ 50% faster than before (confirmed by 5 runs on 10K typical records from our FeatureCenter and MainDemo apps).
*Exact percentage in your end application may vary depending on the complexity of a business model, the number of processed records and other factors.

    created 11 years ago (modified 11 years ago)

    The XPObjectSpace class was improved to speed up CRUD manipulations with large amount of data, e.g. for data import scenarios. In particular, creation and update operations are now ~ 50% faster than before (confirmed by 5 runs on 10K typical records from our FeatureCenter and MainDemo apps).
    *Exact percentage in your end application may vary depending on the complexity of a business model, the number of processed records and other factors.

      Comments (2)

        Thanks Dennis! Can you already estimate about which version this will be included?

        Dennis Garavsky (DevExpress) 11 years ago

          You are always welcome, Kim!
          This improvement should be available with version 13.2.

          created 11 years ago (modified 11 years ago)

          Hello Frank,
          Thank you for your ideas. We are aware of this limitation, and calling the CommitChanges method for smaller objects portions is the recommended way to work around it. To improve performance more significantly, you can create a separate UnitOfWork and objects in it. However, note that since the UnitOfWork has the objects identity map, it is also recommended to commit large transactions in portions.
          I have passed your ideas regarding the XPObjectSpace improvements to our developers. We will take them into account in the future.

            Comments (3)

              Hello Anatol,
              we have fixed the problems ourselves. We have a custom XPObjectSpace derivation and we have injected the following class within the Constructor into the "objectsToSave" field.
              MyXPObjectSpace {
              this.objectsToSave = new HashedArrayListForObjectSpace();
              }
              We got a huge performance gain with this optimization.
              So you should definitely consider using a HashSet or some aquivalent within the XPObjectSpace for performance reasons.

              Dennis Garavsky (DevExpress) 11 years ago

                Thanks for sharing your solution here. We will take it into account for the future.

                Apostolis Bekiaris (DevExpress) 11 years ago

                  Thanks from eXpandFramework as well, your idea has been implemented in version 13.1.6.19

                  Disclaimer: The information provided on DevExpress.com and affiliated web properties (including the DevExpress Support Center) is provided "as is" without warranty of any kind. Developer Express Inc disclaims all warranties, either express or implied, including the warranties of merchantability and fitness for a particular purpose. Please refer to the DevExpress.com Website Terms of Use for more information in this regard.

                  Confidential Information: Developer Express Inc does not wish to receive, will not act to procure, nor will it solicit, confidential or proprietary materials and information from you through the DevExpress Support Center or its web properties. Any and all materials or information divulged during chats, email communications, online discussions, Support Center tickets, or made available to Developer Express Inc in any manner will be deemed NOT to be confidential by Developer Express Inc. Please refer to the DevExpress.com Website Terms of Use for more information in this regard.