API stops after 65000 records
Hi
We have a sql datawarehouse setup and every night there is a process where a service connects to the SO API and synchronizes our data. Last night there was an interruption after 65000 records in the Appointment table. Projects, Contacts and Users didnt give an error. I started the sync twice now and each time the sync stops after 65000 records in the Appointment table and log the following error:
System.Net.Http.HttpRequestException: An error occurred while sending the request. ---> System.Net.WebException: The remote name could not be resolved: 'online4.superoffice.com'
I tried checking with Postman if there was an error but also there I notice something different. I check the TenantStatus to get the correct API url; in our case thats online4.superoffice.com/... but when I use that API with de SO-AppToken and SOTicket in the header (like we always do) we now get a 401 Unauthorized error.
Is anyone aware of any changes to the API or why the sync systematically breaks after 65000 records?
Any help appreciated! Thanks in advance
All Replies (4)
Are you retrieving all these records at once? or over multiple api calls? Maybe the result size is too big, causing you to get a timeout error? Try decreasing the batch size.
(Note: if you are retrieving this many records, maybe database mirroring is a better option?)
I updated the batch size to 500 and now it stopped twice after exactly 7000 records.
Hi Eddy,
I see the API calls you're making and I would sugguest optimizing even more these requests: https://docs.superoffice.com/en/developer-portal/best-practices/index.html#optimization
When you get into the higher number it may be an idea to include like a give me a batch from the last primaryKey order by primaryKey.
Do contact us at appdev@superoffice.com if you want more insight than what you get in the developer portal under metrics: https://docs.superoffice.com/en/developer-portal/analytics/metered-services.html