We’ve developed some resources to help you work effectively from home during COVID-19 Click to learn more

SuperOffice performance issues possibly due to sentry user preferences?

Hi all

I'm sitting with a customer, and we are both having quite a hard time figuring out why the performance has dropped dramatically in their environment to the point where SuperOffice crashes several times during the day for pretty much the whole user base. Generally though, the pattern we observe is that the heavy users does experience more crashes than those who are not using SuperOffice that intensively. Obviously. There are no specific pattern of why the client is crashing though, other than this specific error message, which pointed us in two directions: RAM usage, and sentry preferences.

180820 10:56:32 CZBN 3.1001[SOCRM.exe] , Error calling: class NSPreviewProvider::NSPreviewData __stdcall NSPreviewProvider::_GetPreviewData(const class SvStr &) throw(class SException)
ContactSentry failed to compute table rights
Failed to Validate cache: SentryPreferenceCache
Failed to Refresh cache: SentryPreferenceCache
Exception of type 'System.OutOfMemoryException' was thrown., Level: Error
At: 10:56:32

Element:
Message: ContactSentry failed to compute table rights
Type: SuperOffice.Exceptions.SoSentryException
Details:
at SuperOffice.CRM.Security.Sentry`1.SuperComputeTableRights()
at SuperOffice.CRM.Security.Sentry.GetTableRight(TableInfo table)
at SuperOffice.CRM.Security.SentryCollection.TableRight(TableInfo table)
at SuperOffice.CRM.Security.SentryCollection.CanTableDo(TableInfo table, ETableRight rights)
at SuperOffice.Data.SoDataReader.InnerRead()
at SuperOffice.Data.SoDataReader.Read()
at SuperOffice.CRM.Previews.ContactPreviewPlugin.InnerGetPreview(PreviewData previewData)
at SuperOffice.CRM.Previews.PreviewPluginBase.GetPreview(Dictionary`2 hints, PreviewData previewData)
at SuperOffice.CRM.Previews.PreviewProvider.ProcessHints(String nameValueHints)
at NSPreviewProvider._GetPreviewData(NSPreviewData* , SvStr* i_PreviewHint)

Inner Element:
Message: Failed to Validate cache: SentryPreferenceCache
Type: SuperOffice.Exceptions.SoException
Details:
at SuperOffice.Data.Cache.CacheBase.<Validate>b__2()
at SuperOffice.Util.ReaderWriteLockExtensions.WithWriterLock(ReaderWriterLock theLock, Action body)
at SuperOffice.Data.Cache.CacheBase.Validate()
at SuperOffice.Data.Cache.SoCache.GetCache(Type cacheType)
at SuperOffice.CRM.Security.SentryPreferenceCache.GetCurrent()
at SuperOffice.CRM.Security.Sentry`1.SuperComputeTableRights()

Inner Element:
Message: Failed to Refresh cache: SentryPreferenceCache
Type: SuperOffice.Exceptions.SoException
Details:
at SuperOffice.Data.Cache.CacheBase.<Refresh>b__3()
at SuperOffice.Util.ReaderWriteLockExtensions.WithWriterLock(ReaderWriterLock theLock, Action body)
at SuperOffice.Data.Cache.CacheBase.Refresh()
at SuperOffice.Data.Cache.CacheBase.<Validate>b__2()

Inner Element:
Message: Exception of type 'System.OutOfMemoryException' was thrown.
Type: System.OutOfMemoryException
Details:
at SuperOffice.Data.Dialect.Dialect.ConvertFromDbType(Object value, FieldDataType dataType)
at SuperOffice.Data.Dialect.Dialect.DataReaderGetItem(SoDataReader soReader, IDataReader innerReader, Int32 i)
at SuperOffice.Data.Dialect.Dialect.DataReaderGetValue(SoDataReader soReader, IDataReader innerReader, Int32 i)
at SuperOffice.Data.SoDataReader.get_Item(Int32 i)
at SuperOffice.Data.SoDataReader.get_Item(FieldInfo fieldInfo)
at SuperOffice.CRM.Rows.UserPreferenceRow.OnLoad(SoDataReader reader, TableInfo tableInfo)
at SuperOffice.CRM.Rows.TableRowBase.Load(IdxBase index, SoDataReader reader)
at SuperOffice.CRM.Rows.TableRowsBase.RowsLoad(ITableRowLoadHandlerFactory tableRowHandlerFactory)
at SuperOffice.CRM.Rows.PrivateFactory.PrivateUserPreferenceRowsFactory.SuperOffice.Factory.IPrivateFactory.Create(Type type, Type[] constructorArgumentTypes, Object[] constructorArguments)
at SuperOffice.Factory.TypeFactories.Create(Type type, Type[] constructorArgumentTypes, Object[] constructorArguments)
at SuperOffice.Factory.ClassFactory.Create(Type type, Type[] constructorArgumentTypes, Object[] constructorArguments)
at SuperOffice.CRM.Rows.UserPreferenceRow.UserPreferenceRowIdxBase.ToUserPreferenceRows(ITableRowLoadHandlerFactory tableRowHandlerFactory)
at SuperOffice.CRM.Security.SentryPreferenceCache.OnRefresh()
at SuperOffice.Data.Cache.CacheBase.<Refresh>b__3()

In addition to the windows client crashing, the IIS server is also under heavy workload to the point where it simply just stops responding at some point, which is where we will need to perform a IIS reset in order to get it up and running again. Datadirector from Amesto throws the same error as above.

 

So we have been doing some diagnosis for quite some time now, and we've come up with a pretty specific theory of why SuperOffice behaves the way it does, and it revolves around the sentry preferences...

Back in May 2018, i developed a script for the customer, that took advantage of setting sentry preferences for individual appointments inside their SuperOffice. Their need for control of appointments made 3rd party software as IB DataDirector overkill, so they were happy when i told them about sentry preferences in SuperOffice

(Per this article: https://community.superoffice.com/en/content/content/client-sdk/setting-data-rights-with-sentry-and-userpreferences/ )

So their wish was that a certain group inside the organisation should be able to create appointments where only the type should be visible for the rest of the organization and not the text. So i constructed a relatively simple windows script for that purpose, which i have included in this thread.

So the solution was working just the way it should, and they've been pretty happy with it so far.

Since may this solution has generated around 60k userpreferences in the userpreference table, and we observed that it has a memory impact of around 300 mb on each of the windows clients compared to a similar database where the sentry preferences has been cleaned. 

 

So our question is if the sentry preferences are loaded all at once during start up? The scenario that we hope for is that you'll answer they are computed on demand or loaded into some kind of cache so that they don't interfere with the general performance or memory footprint of the client.

 

The customer is running 8.0 SR4 and they are primarily using the windows client. The web server is used for the Marketing modules

 

I have attached the script we're using here:

 

Sub OnCurrentAppointmentBeforeSave
Dim app
Dim associate
Set app = CurrentAppointment
Set associate = CurrentDiaryOwnerAssociate
Dim groups
Dim GroupSetting
GroupSetting = "Personalejura"
Dim IsInGroup
Dim IsSecure
IsSecure = "false"
IsInGroup = false
Dim group_id
Set groups = associate.Groups
For Each group In groups
With group
if .Text = GroupSetting then
IsInGroup = true
group_id = .ID
end if
End With
Next
IsSecure = Database.Preferences.Get("Rights-appointment-" & CStr(app.Identity), "Text.text", "",2)
if IsInGroup then

if IsSecure = "0" then

If app.UDef.ByProgId("SuperOffice:4").value <> 12507 And app.UDef.ByProgId("SuperOffice:4").value <> 12508 then
'Set sentry
app.UDef.ByProgId("SuperOffice:4").value = 12507 'Sæt til Ja hvis ellers fortrolighed ikke er udfyldt
else
if app.UDef.ByProgId("SuperOffice:4").value = 12507 then
answer = MsgBox("Vil du bibeholde aktivitetens fortrolighed", vbYesNo + vbQuestion)
If answer = vbNo Then
'Remove sentry

app.UDef.ByProgId("SuperOffice:4").value = 12508 'Set to no

end if
end if
end if
else
If app.UDef.ByProgId("SuperOffice:4").value <> 12507 And app.UDef.ByProgId("SuperOffice:4").value <> 12508 then
'Set sentry

app.UDef.ByProgId("SuperOffice:4").value = 12507 'Set to yes
else
answer = MsgBox("Vil du aktivere fortrolighed på denne aktivitet? Andre medarbejdere vil således kun være i stand til at se typen på aktiviteten medmindre de selv er en personalejuridisk medarbejder.", vbYesNo + vbQuestion)
If answer = vbYes Then
'Set sentry

app.UDef.ByProgId("SuperOffice:4").value = 12507 'Set to yes
else
app.UDef.ByProgId("SuperOffice:4").value = 12508 'Set to no
end if
end if
end if
end if
End Sub
Sub OnCurrentAppointmentSaved
Dim app
Dim associate
Set app = CurrentAppointment
Set associate = CurrentDiaryOwnerAssociate
Dim groups
Dim GroupSetting
GroupSetting = "Personalejura"
Dim IsInGroup
Dim IsSecure
IsSecure = "false"
IsInGroup = false
Dim group_id
Set groups = associate.Groups
For Each group In groups
With group
if .Text = GroupSetting then
IsInGroup = true
group_id = .ID
end if
End With
Next
IsSecure = Database.Preferences.Get("Rights-appointment-" & CStr(app.Identity), "Text.text", "",2)
FortroligUdef = app.UDef.ByProgId("SuperOffice:4").value
if IsInGroup then

if IsSecure = "0" then

If FortroligUdef = 12508 Then
'Fjern fortrolighed på aktivitet
Database.Preferences.Set "Rights-appointment-" & CStr(app.Identity), "Text.text", "63", 2 'read / write access all
Database.Preferences.Set "Rights-appointment-" & CStr(app.Identity), "Table", "63", 2 'read / write access for all
' msgbox("Aktiviteten er ikke længere fortrolig!")
elseif FortroligUdef = 12507 Then
Database.Preferences.Set "Rights-appointment-" & CStr(app.Identity), "Table", "17", 2 ' read access for all 
Database.Preferences.Set "Rights-appointment-" & CStr(app.Identity), "Table", "63", 4 ' read / write access for the group of the creator
Database.Preferences.Set "Rights-appointment-" & CStr(app.Identity), "Text.text", "0", 2 ' no access for all to see text
Database.Preferences.Set "Rights-appointment-" & CStr(app.Identity), "Text.text", "63", 4 ' Read access for the group of the user
' msgbox("Aktiviteten er nu fortrolig!")
end if
else
'Kig på udef felt om det er sat til ja eller nej
If FortroligUdef = 12507 Then
'Set sentry
Database.Preferences.Set "Rights-appointment-" & CStr(app.Identity), "Table", "17", 2 ' read access for all 
Database.Preferences.Set "Rights-appointment-" & CStr(app.Identity), "Table", "63", 4 ' read / write access for the group of the creator
Database.Preferences.Set "Rights-appointment-" & CStr(app.Identity), "Text.text", "0", 2 ' no access for all to see text
Database.Preferences.Set "Rights-appointment-" & CStr(app.Identity), "Text.text", "63", 4 ' Read access for the group of the user

' msgbox("Aktiviteten er nu fortrolig!")
end if
end if
end if
End Sub
Sub OnViewShown( hWnd, fullPath )
If fullPath = "MainWindow.AppointmentDialog" then
Dim app
Set app = CurrentAppointment
Dim IsSecure

IsSecure = "false"

IsSecure = Database.Preferences.Get("Rights-appointment-" & CStr(app.Identity), "Text.text")
if IsSecure = 0 And app.Identity <> 0 then
CurrentAppointment.Description = ""
if app.Associate.Person.Emails.Empty then
MsgBox("Denne aktivitet er fortrolig. Kontakt " & app.Associate.Person.FullName & " for yderligere information om denne aktivitet.")
else
Dim test
For Each email In app.Associate.Person.Emails
if test = "" then
test = email.Address
end if
Next
MsgBox("Denne aktivitet er fortrolig. Kontakt " & app.Associate.Person.FullName & " på " & test & " for yderligere information om denne aktivitet.")

end if
end if
end if
end sub

 

 

 

 

 

 

RE: SuperOffice performance issues possibly due to sentry user preferences?

So you're saying that you have 60.000 rows in crm7.userpreference for setting Sentry rights?

And in WEB you have no scripts, but it still crashes?

Gut feeling is that that mechanism was never made to handle 60.000 rows. Solution is most likely to write a proper Sentry Plugin which calculates the rights on the fly.

Av: Frode Lillerud 30. aug 2018

RE: SuperOffice performance issues possibly due to sentry user preferences?

Userpreference is loaded during startup, and that many rows will kill it yes.

I actually found this from way-back-when we did not clean up correctly the old counter preferences when we moved them to countervalue table: https://community.superoffice.com/en/Technical/Forum/Rooms/Topic/?clubId=2&topicId=10915

Av: Margrethe Romnes 30. aug 2018

RE: SuperOffice performance issues possibly due to sentry user preferences?

Well that was in short our suspicion too, after doing some digging as i wrote :(

 

We did consider writing a sentry plugin at first, but since the customer is in the process of considering switching to online in a hopefully foreseeable future, we chose this approach as that was possible to replicate in crm script, and thereby run in online. Unfortunately this particular case has made that wish take quite a leap back in progress.

 

So the next question is, is there any way to get sentry functionality into online, since sentry preferences and thereby a solution that is powered by crm script is no option now?

Av: Dennis Aagaard Mortensgaard 30. aug 2018

RE: SuperOffice performance issues possibly due to sentry user preferences?

System.OutOfMemoryException. 

This is an exception from our In-Process Netserver.
Once this happens, all bets are off.
Easiest way to get here is to try and load close to 100 thousand rows in some archive.
Win has a way of trying to recover from an out-of-memory case but the netserver c# universe has not.

- Win caches your entire userpreferences!
Unfortunately, Win is a hybrid and every time you make a change, Win now also has to tell Netserver to flush it's cache.
Adding preferences dynamically as your script requires will cause a lot of flushing..

- The sentry system using preferences is a "poor-man's sentry plug-in", my 2 cents, obviously. 

Turn on the Script Utils menu for any of these users, "Enable script developer mode" and restart.
Let the user disable all events and see what happens.

 

 

Conrad

 



Av: Conrad Weyns 30. aug 2018

RE: SuperOffice performance issues possibly due to sentry user preferences?

The build-in userpreference based sentry is ideal in situations where you want to protect something for a certain group or user, but the criteria for that 'something' needs to be static. Avoiding that limit by creating preferences for every entity will indeed be destructive as this whole lists needs to be kept in memory and parsed all the time.

For dynamic handling of criteria to apply rights you definately need a custom sentry plugin. Or you can use InfoBridge DataDirector that works as a hybrid solution by threating your rules as dynamic queries against the SuperOffice database, you could send us a support e-mail for more information.

Sentry plugins for online like onprem will never happen since that will be a major security hole as the code will be able to access the whole process/server. Since CRMscript is here now, perhaps in the future we will be able to setup some rights with advanced dynamic lookup using CRMscript?

Av: Carlo Pompen 30. aug 2018