How to migrate documents to SharePoint?

How to migrate documents to SharePoint?

This article will walk you through how to migrate documents to SharePoint.

Microsoft SharePoint Online have a different backup regime than SuperOffice CRM Online have!

As a platform, Office 365 is secure, but your data is not backed up in a way that most organisations require. Typically, there's only a 30-day retention period inbuilt into Office 365, and Microsoft SharePoint Online is only backed up every 12 hours, with just a 14-day retention period

As per Microsoft Online documentation "SharePoint Online retains backups of all content for 14 additional days beyond actual deletion. If content cannot be restored via the Recycle Bin or Files Restore, an administrator can contact Microsoft Support to request a restore any time inside the 14 day window. Microsoft 365 backs up your SharePoint site every 12 hours"


technet.microsoft.com

Requirements

• SuperOffice Online tenant
• 'SuperOffice SharePoint Documents' needs to be fully configured before you can migrate to SharePoint.
  • All SuperOffice users with a user plan: Must be using IDP authentication
  • All SuperOffice users with a user plan: Must log into CRM Online AFTER configuration (For the migration-tool to be able to create documents in sharepoint on behalf of the user)
  • All SuperOffice users need to be a AD-user in the target MS subscription: external AD user from other subscription is currently not supported by the MS API (User Story 39490)
• Authenticate SuperOffice as a trusted app
  • (This app need access to 'Global Admin' rights in O365 during first time setup (to be able to authenticate SuperOffice as a trusted app in your MS 365 solution) - either during setup or send a link to someone who is)
  • This app has already been approved if customerr is using Visible for (Groups and ..)​ when using 'SuperOffice SharePoint Documents'

 

Process Planning

In situations where you are to migrate a CRM Online-tenant where there are a lot of documents (especially old documents) - there are scenarios where it might be cumbersome to plan for all eventualities and make all needed preparations up front.

• this will result in error-situations and unplanned events - which then leads to documents fail to be migrated on the first run.
• the process it self - of migrating documents is somewhat time-consuming - when both the data (document itself) and the meta data needs to be transferred.
  • usually it is the API of SharePoint being the limitation factor of setting these restrictions on speed and synchronous file-transfer. If you plan to migrate thousands of documents - it will take time, perhaps even days to complete.

It is therefor smart to take this into account when you plan the project to migrate many documents.

• Documents that are successfully moved will be deleted from the source library​.
• Documents that can not be moved will be available as before​ (from CRM Online storage).

If a document fails to be migrated - it will be left as is in SuperOffice CRM Online, and can be re-migrated later/next time (ie. after you have made corrections or planned and prepared better). Also time-planning for corrections before next run can be smart:

Normal process
Preparations ➡️ Run Test migration of a few documents ➡️ Analyse report (send to SO R&D, receive guidence from SO R&D) ➡️ Make corrections and preparations ➡️ Run migration ➡️ Analyse report (send to SO R&D, receive guidence from SO R&D) ➡️ Make corrections and preparations ➡️ Run migration

 

Important notes on the pilot feature:

• You may select how many documents to be migrated per migration-job by creating a document-selection.
  • it is not supported to use Owner Current user as criteria for a dynamic document-selection.
• Only support for users that has Microsoft login and that all users belongs to the same MS Domain
  • Users from other vendors / password login will not be supported.
    • those user must be set to former employee during configuration.​
  • We recommend migrating documents to SharePoint after all users have logged in to Online SuperOffice CRM after SharePoint is configured. Otherwise, we will not be able to create the document with the correct owner in SharePoint.​
    • If not successful, this test will cause an error and you will not be allowed to proceed with the configuration.
  • We check that all users belongs to the same MS Domain. If they do not, that could cause issues when applying permissions to or creating SharePoint Documents​
    • If not successful, this test will cause a warning, but you will still be able to proceed with the configuration​
• When documents are moved, templates will also be moved. All of them.
• Documents owned by users that does not exist in SharePoint, will get a new owner - SharePoint App (you can choose not to move such documents)​.
• These documents will also be in the folder "Other" if you have chosen Folder structure – user group.
• In order to moved these documents, we need a Global Administrator to authorize 'SuperOffice Documents library app'.
  • This app has already been approved if customers are using Visible for (Groups and ..)​
• Documents that are successfully moved will be deleted from the source library​.
• Documents that can not be moved will be available as before​ (from CRM Online storage).
• Limitation: On older MS Office files:
  • There is a limitation in how the new SharePoint integration handle legacy office files of the old binary type (.doc, .xls, .ppt)​
    • This is due to limitation in the Graph api and SharePoint itself.
    • Editing of this type of document cannot be done with Office Online, it has to be done with the desktop app. 
    • We strongly recommend that you replace your old legacy template with a template based on the current format.
• For those who have set up and use "Visible for" in SuperOffice
  • Advanced Data Access Options 
    • Visible for me -> private in Sharepoint
    • Visible for all -> default access rights from Sharepoint
    • Visible for group -> private in Sharepoint

 

Preparations

1.  Migration tool need access to target library (Access Token Error 400)
   • All users with a user plan and who owns a document which should be migrated:
     • All users with a user plan need to log in via online.superoffice.com AFTER "SharePoint Documents" is configured.
       • The migration tool uses the users MS access token to create and store the document in SharePoint.
         (Superoffice start to store crendentials on login from the point in time "SharePoint Documents" is configured)
2. User need accss to SharePoint (Error 404)
   • All users with a user plan and who owns a document which should be migrated:
     • User need to be using IDP authentication towards the SharePoint-target (SuperId password is not sufficient)
     • User need to be a AAD user in SharePoint-target (access right is not sufficient)
3. Retired users need access to target library
   • Old users / retired users (no user plan) do not have access to SharePoint via 1. or 2. above.
     • Documents owned by such users, it requires 'SuperOffice Documents library app' to migrate these documents.
       • To authorize 'SuperOffice Documents library app' you need access to 'Global Admin' rights in O365 to be able to authenticate SuperOffice as a trusted app in your MS 365 solution
4. Missing files (Error Not found in the document archive)
   • Customer who has migrated from onsite, or have done manual cleanup (ie. CRM script etc.),
     may have files no longer stored in source (CRM Online).
     • The files needs to be opened from the CRM client - to also be migrated
     • To check if the file exist in source (CRM Online), you may download the archive using the SuperOffice Download tool
5. Missing content in files (Error Bad Request - Value cannot be NULL)
   • Customer who has migrated from onsite, or have done manual changes to files - or is corrupted for some reason
     may have files no longer possible to open or use
     • The files needs to be possible opened from the CRM client - to also be migrated.
     • To check if the file exist in source (CRM Online), you may download the archive using the SuperOffice Download tool
6. Access to files
   • If there is set "visible for" on a file - it requires 'SuperOffice Documents library app' to migrate these documents.
7.  File with same filename (Error 409 Conflict)
   • It might happen based upon how you have configured the folder structure in the SharePoint-target
     • If 2nd. file is migrated to same folder with same name - it will result in an error "409 Conflict". 
       • File would then need to be manually renamed before your net migration job. Contact support if needed.
8.  Permission denied while creating document! Make sure user has access to the configured document-library and try again!
   • Most likely the SharePoint user-account do not have sufficient rights to create the file in that folder. Check your SharePoint setup. Ask your SharePoint admin for help.
9. Unable to find document
   • When "Visible for" is set on the document to a (SuperOffice) group that the user no longer belongs to (and which has actually been deleted in SuperOffice admin).
     
     • Documents properties need then to be change to a existing group or to "all" - or restore the old group in SuperOffice admin.
10. Limitation in how SharePoint handle legacy office files of the old binary type (.doc, .xls, .ppt)​
    • This is due to limitation in the Graph api and SharePoint itself
    • Editing of this type of document cannot be done with Office Online, it has to be done with the desktop app. ​
      • We strongly recommend that you replace your old legacy template with a template based on the current format. ​
    • Creating a document from CRM works fine, but when you try to edit the document, you will notice that you cannot do so.
      • You must use Edit document and Open in Desktop App
        • NOTE: Do not use Edit and then convert, that will create a copy of the document and next time you open it from CRM the changes aren’t there.
    • Opening a document from CRM works fine, but when you try to edit the document, you will notice that you cannot do so.
      • You must use Edit document and Open in Desktop App
        • NOTE: Do not use Edit and then convert, that will create a copy of the document and next time you open it from CRM the changes aren’t there.
        • NOTE: The same applies if you open a document from within SharePoint​
          
           

          ​

 

Admin Client​ - start the migration job

SuperOffice SharePoint Documents needs to be fully configured before you can migrate to SharePoint.

If not all preparations are completed - most likely there will be documents failing to be migrated.

Once the pilot feature is activated, the SuperOffice Admin will get a new option in "Document library, under "Preferences" - to start the migration wizard.

The configuration in the SuperOffice Admin is point-and-click. This is part i primarely to select the values of wich documents you would like to migrate

You click the text to get started:

Overview

In the overview you can see ongoing and previous jobs. You can also see details for these jobs (se separate slide for the details view)​

•  Date – Date when the job was created​

• Status​

  • NotStarted – the job is preparing and not started yet​

  • Templates Started  – Moving of templates has started​

  • Documents Started – Moving of documents have started​

  • Cancellation In Progress – Somebody has stopped the job, and the job needs to finish ongoing migration before stopping. 

  • Cancelled – The job has been cancelled​

  • Completed – The job has moved all documents (and templates) with success​

  • Completed With Failures – one or more documents failed, but the job completed. ​

  • Failed – For some unknown reason, the job failed.

• Start – Date and time the job started​

• End – Date and time when the job ended, shows a progress bar during ongoing job​

• #moved – Counter for number of documents / templates that has been moved​

• Source – show the selected source from Wizard Step 1 ​

• Details – “Details” button​

  • Light bluish – Ok​

  • Light orange – some failures​

  • Light red – failed​

  • Click on the button to see details​

• “Move documents” – starts the wizard. It will not be possible to start a new migration during running migration jobs​

 

Overview - Details view​

Summary section​

Shows the summary of the scheduled using the wizard.

​Status section​

Shows the status for ongoing or completed job​

• Shows the status for ongoing or completed job​

  • Status – “status”, where “status” is the same as in the status column in overview.
  ​

Migration:  Wizard​

General about the Wizard​

The wizard has 3 steps:​

​Migration:  Step 1 – Select documents ​

​Migration:  Step 2 – Review and test​

​Migration:  Step 3 – Confirm and schedule job

• ​Each of these steps must be filled in or passed before you can move to the next step. The “Next step” button will be disabled until you have finished the step. ​
• “Previous step” button takes you back one step. ​
• “Cancel” button takes you back to the Status page. ​
• In Step 3 you can start the job​

 

​Migration:  Step 1 – Select documents ​

Pilot Limitation: Maximum 20 000 documents can be migrated per migration-job.

A document-selection can either be used to ensure you don't exceed the limitation or as the source of which documents to migrate in a specific migration-job.
If you plan to migrate more documents than this limit - you would need to execute multiple migration-jobs, each within the limit.

In step one you select the documents you want to move from your “old” library to your current library.

Date for “All documents created after:” defaults to 1st of January last year. ​

If you don’t want to move documents belonging to former employees – you can skip those by ticking on the “Skip…”​

See also general information​

​

 

​Migration:  Step 2 – Review and test​

Based on what was selected in Step 1, the wizard will run through some tests to make sure it will be able to move documents. ​

If one of these tests failed, we will provide you with an explanation and how to fix it​

• Analyzing documents to be moved: ​
  • Identifying number of documents​
    • Counting all documents based on step 1​
  • Identifying number of templates​
    • First time, all templates will be counted​
  • Access to target document library​
    • Sufficient access in order to move documents to the new library​
  • Access to target template library​
    • Sufficient access in order to move templates to the new library​

• Analyzing documents to be moved: ​Testing user credentials​
  • Checks if the we have credentials for all user in the selected documents​
  • Testing access for users without credentials (app permission)​
  • If moving documents with no credentials (ref Step 1 – Not skipping documents), authorizing an app for moving such documents will be needed (if not already authorized)​
  • Testing access rights for user groups (visible for)​
  • Checking if you have enabled visible for. Then testing that user-groups used by the documents being migrated are mapped to SharePoint domain groups​
• Identifying documents to migrate.​

The result set after removing documents belonging to users without credential (if that is ticked) and documents belonging to users without credentials and visible for on document is set to “me” ​

• Documents to be moved - estimates: ​
  • Total number of documents​
    • From step 1 – Selected documents​
  • Documents omitted​
    • Documents belonging to users without credentials and visible for on document is set to “me” are omitted. ​
    • Documents belonging to users without credential (if that is ticked) is omitted. ​
  • Documents already moved​
    • Documents already in your new library​
  • Documents to be moved​
    • The number of documents that will actually be moved​

 

​Migration:  Step 3 – Confirm and schedule job

• Summary section​
  • Gives a summary of the job​
• Notes section​
  • We ask you to kindly notice these Notes​
• Button – Start job now​
  • This button will start the migration job and you will return to the overview page and can see the progression of the job. ​

After you have migrated the documents, you will then open, edit your existing SuperOffice documents in Office365 SharePoint directly from SuperOffice. These documents can also be accessed by everyone in your company (if set up to do so), even those who do not use SuperOffice.

 

Job Details

If documents fail to be migrated - there is a error log available to download. Send this to pilot@ and we can try to assist.