We’ve developed some resources to help you work effectively from home during COVID-19 Click to learn more

Simple CRMScript that triggers Memory error in SOD

Hi,

we're seeing memory limitation issues every now and then when creating scripts, particularly when working with files/documents.

I've isolated the error into this simple script. It crashes in SOD, but not in Online. I assume because the memory limitation is higher in Online.

#setLanguageLevel 3;

String url = "https://upload.wikimedia.org/wikipedia/commons/2/2d/Snake_River_%285mb%29.jpg";

HTTP http;
Byte[] response = http.get(url);

Integer i = 10;

This script downloads a file that is about 5MB, and puts the result into a Byte array.

In SOD it crashes with:

EjScript::RunTimeException: Script is taking too much memory: 247008Kb

247MB of RAM seems a bit excessive when downloading 5MB...?

Also note, that the last line in the script needs to be there to trigger the error, for some weird reason.

 

I'm thinking there might be some sub-optimal memory management happening in the background here?

RE: Simple CRMScript that triggers Memory error in SOD

Hi Frode,

i think there is a 5MB limit because 4MB is working fine:

http://mitch.cosbuild.net/wp-content/uploads/2015/11/paris-christmas-orig-4mb.jpg

 

Dennis

Av: Dennis Pabst 18. okt 2019

RE: Simple CRMScript that triggers Memory error in SOD

Yes, but 4 or 5 megabytes is still far from the 247MB it is saying the script is using.

Av: Frode Lillerud 18. okt 2019

RE: Simple CRMScript that triggers Memory error in SOD

I have reproduced that the process is using that amount of memory, so seems like the reporting is correct. Theory is that is has something to do with the HTTP class (or cURL which is the underlying comm component).

The reason for not getting the exception without the "Integer" line, is that a new statement after the HTTP get is required to get a new memory measurement. So any line after the http.get() will cause the exception to appear.

I'll post more info when deeper knowledge is aquired :)

Av: Stian Andre Olsen 18. okt 2019

RE: Simple CRMScript that triggers Memory error in SOD

The problem is that the HTTP.get() methods returns an array of bytes, and in your case that will be an array of over 5 million elements (one for each byte). In CRMScript, there is some memory (and performance) overhead on arrays.

The solution will probably be to create a new method on the HTTP class that returns a single object, containing the result of the get() call. 

This way we do not break old CRMScript implementations, so to take advantage of the improved version, one will have to exchange the methods where having this problem.

Av: Stian Andre Olsen 18. okt 2019

RE: Simple CRMScript that triggers Memory error in SOD

Great, thanks for looking into this. We've had an increasing number of related issued with all the Online-customizations we're working on, so eliminating this issue will be very helpful.

Would it make sense to get it to return a NSByteStream instead?

Av: Frode Lillerud 18. okt 2019

RE: Simple CRMScript that triggers Memory error in SOD

I think it is called "NSStream", but that seems like a good suggestion.

Av: Stian Andre Olsen 21. okt 2019

RE: Simple CRMScript that triggers Memory error in SOD

For future reference: https://community.superoffice.com/en/developer/forum/rooms/topic/gen-dev-group/gen-dev-annoucements/support-for-nsstream-instead-of-byte/

#setLanguageLevel 3;

String url = "https://effigis.com/wp-content/uploads/2015/02/DigitalGlobe_WorldView2_50cm_8bit_Pansharpened_RGB_DRA_Rome_Italy_2009DEC10_8bits_sub_r_1.jpg"; //25 MB

HTTP http;
NSStream stream = http.getAsStream(url);
String base64 = encodeBase64(stream);

Attachment a;
a.setValue("name", "big_image.jpg");
a.save(); 
a.saveBase64(base64);

String downloadUrl = a.getDownloadUrl(true, false);
printLine(downloadUrl);
Av: Frode Lillerud 9. jan 2020