Coming soon: Your brand new Help Center & Community! Get a sneak-peek here

Problems when importing large dataset with NetServer API

Hi,

We have developed import-program for our customer, program reads data from csv-file and then imports it in SuperOffice database with NetServer API.

Some simple pseudocode for operation:

using (SoSession session = SoSession.Authenticate(user, password))

{

  for(int i = 0; i < 5000; i++)
  {

    Contact contact = null;
    contact = Contact.CreateNew();
    contact.SetDefaults();

    contact.Number2 = customerNumber;
    contact.Name = name;

    //Main phone
    contact.Phones.Delete();
    contact.Phones.Clear();
    PhoneRow phone1 = PhoneRow.CreateNew();
    phone1.SetDefaults();
    phone1.Phone = FormatPhoneNumber(phoneNumber);
    phone1.Rank = 1;
    phone1.Save();
    contact.Phones.Add(phone1);

    contact.Save();
  }

}

There might be thousands of customers to import and operation can take 10 minutes. It's good enough performance as program runs only one time at every night.

Now program seems to work fine by itself but problems arise when our customer have InfoBridge Synchronizer working at the same time. IB Synchronizer crashes when this program performs import.

So is this import-program effectively blocking those database tables what it updates? Or are there some other problematic areas in code to look into? Like should every update performed in it's own session?

RE: Problems when importing large dataset with NetServer API

Hi MIkko,

The higher you are in the API, the slower things are going to seem. Using business object is good when you want a simple API to perform quick tasks easily. When you want better performance, you want to got down to use either Rows, or even faster OSQL.

In the article What is NetServer Part I, I mention the benefits of using soSession.Suspend and SoSession.Continue, to avoid keeping a session open, and simplifying authentication for subsequent operations. There is a 24h timeout to be aware of, so if you are only doing this nightly, then authentication each time shouldn't be an issue. 

Hope this helps!

Af: Tony Yates 15. sep 2017

RE: Problems when importing large dataset with NetServer API

You are showing "pseudo code" so this is probalby a shot in the dark:

How about adding a Sleep( ... ) in your loop?

/conrad

Af: Conrad Weyns 15. sep 2017

RE: Problems when importing large dataset with NetServer API

Thank you both for answers. 

Sleep might be worth trying as program at the moment does it's operations "full speed".

Af: Mikko Tillikainen 15. sep 2017