READ ME

​

We’re here to support you! If you have questions not addressed here or would like a web conference walkthrough, please reach out to us at support@civiclens.com. We’ll get back to you within one business day.

 

If you have issues unzipping items, or feature service FGDB backups seem empty

7-Zip is recommended for extracting backups. The default Windows extractor sometimes doesn’t like the long names/paths of backup files, and will either throw an error when extracting, or will appear to extract successfully but the extracted FGDB will be incomplete and therefore appear to have no data/features. If you are unable to use

7-Zip, you can rename your backup files to a shorter name and/or move them somewhere with a shorter path and extract them there.

 

Consult the status/console window

If you get a “Something Went Wrong” warning, review the status/console window to see what the error was. It’s usually an easy fix such as a typo or missing parameter entry.

 

The status/console window sometimes shows responses from the REST/Python APIs

Occasionally you’ll see automated responses from the Esri APIs in the console window that look like errors (examples include “Token Required”, “Invalid URL”, “Could not generate token…”, “There was an error”). This is expected, and the Backup Utility will try again or use a different approach behind the scenes.

 

Username and password are case sensitive

"Some_User.User" is not equivalent to "some_user.user". Credentials must be from the built-in identity store and should have Administrator privileges. Note: If your password starts with a hyphen (-), you must add a leading space when providing it to the application.

 

Create a new directory for backups

Depending on your settings, Windows may limit programmatic access to certain folders (e.g., Documents, Desktop). Creating a new dedicated directory for your backups will help sidestep folder permissions issues.

 

Hosted tile layers (tile caches and vector tile services) are handled specially

Hosted tile layers are exported to tile packages or vector tile packages; however, if there is an existing tile package item or vector tile package item with an identical title, the service item is skipped to avoid backing up the same data twice. Tile layers with more than 500,000 tiles cannot be backed up due to platform limitations on tile counts for hosted tile layers.

 

If the application appears to be hanging, it’s working on a service in the background

Some feature services (especially large ones with lots of attachments) can take some time while the data is prepared on the server side. The application will move on to the next item after the number of minutes set in the timeout parameter (default is 60), continuing to work on the original service in the background. If any large feature services fail or go to CSV-only, increase the value of the timeout parameter.

 

Portal for ArcGIS only: Be sure you have available space for temp FGDB exports

If you are backing up hosted items from Portal for ArcGIS, be sure there is available disk space on your server to accommodate temporary file geodatabase exports during the process. Temporary exports are removed as soon as they are successfully exported.

 

 

FAQ

 

What item types are included in backups?

Most item types are backed up by the application by default. Exceptions include items that only reference other items and would therefore be duplicative (e.g., layer copies or views), or anything that’s just a pointer to an external application or data. A full list of item types can be found here.

 

Do feature service exports come with attachments, domains, and related tables?

Yes. Feature services are exported as file geodatabase with all attachments, domains, and related tables.

 

Does CivicLens have access to my data or credentials?

No. The application does not communicate with us in any way.

 

Can I run more than one instance of the application simultaneously?

Yes. If you’re using the scheduled version, you’ll need to copy-paste the application and parameters.xlsx file into a new folder for each additional instance you wish to run.

 

If you’re using the on-demand version, you can just double-click it again to open another instance.

 

Can I use the items in my org/portal while the backup is running?

Yes. The backup process will not affect the function of your items.

 

Why do a few feature services get special tags?

The application may occasionally add tags to a small number of feature services that include the text [do_not_remove_this_tag]. These tags help improve performance in subsequent backup runs by sending these services to a special function in the code based on their performance in the previous backup run. They also support the differential backup functionality by marking services for which the settings have been updated but not the data itself. Inadvertent user deletion of these tags will not cause the application to miss any items. If adding tags is unacceptable, let us know and we can configure a no-tag version.

 

NOTE: The ‘Data Last Updated’ date in your portal does not always reflect the actual date of last edits because changes to settings can cause this date to update. The Backup Utility accounts for this (one of the reasons for the occasional tags).

 

What does “Add Item Dependencies” parameter do?

Selecting ‘Yes’ will write item dependencies to the inventory file in the backup folder. In the case of feature services, this is a list of web map item IDs that consume that service; in the case of web maps, it’s a list of feature layer item IDs present in the map, and in the case of dashboards and web mapping applications, it’s the associated web map item ID. Evaluating dependencies will increase backup time.

 

What does the timeout parameter do?

The timeout parameter specifies how long the application will work on a service before starting the next item. When the timeout limit is reached, the service is sent to its own thread and continues to be processed in the background. The 60-minute default value is more than enough for most orgs, but if you have large feature services that fail, increase this value.

 

Can I use a UNC path for the backup location?

Yes.

 

How are items restored?

Check out our Item Restoration Guide.

 

How are Experiences and the new Story Maps handled?

ArcGIS Experience apps and the New Story Maps are built with a new architecture that stores the item resources in the item itself. The Backup Utility does export the resources for these items, as well as the configuration JSON. Note that Story Maps with no resources (e.g., someone created a Story Map but never configured it) or those that have not been published will not back up with resources, instead downloading as JSON-only with notification to that effect.

 

Why won’t my feature service export?

If the feature service is large (multi-GB), try increasing the value of the timeout parameter on the Options tab (Scheduled Version: Cell B19 in parameters.xlsx).

 

Rarely, a feature service won't export as a file geodatabase due to internal issues with the service, even if that service functions as expected in your maps and apps. In our experience, these services will also not be exportable from the item page in your org. If you experience any services that consistently fail on export, let us know and we can help you resolve the issue.

 

TROUBLESHOOTING

 

Please don’t hesitate to contact us at support@civiclens.com. A license purchase includes a small services engagement to get your backups running seamlessly, and unlimited support is included.

 

Authentication issues

1. Be sure both your username and password are entered with proper case. Credentials must be from the built-in identity store.

2. Be sure your ArcGIS Online org URL is entered correctly.

3. You may need to request that your IT department whitelist your ArcGIS Online or Portal URL for internet access.

 

The application looks like it’s hanging/not doing anything

If the application seems to be hanging, it’s either waiting on the export to finish on the server side, or it’s in the process of downloading the export. The process will move on to the next item after the number of minutes set in the timeout parameter on the Options tab (scheduled version: Cell B19 in the parameters file), continuing to work on the long-running item in the background. ​Large feature services with lots of attachments can take an hour or more to prepare on the server side, plus download time.

 

An exception to the timeout is when there are active backups for exceptionally large services still downloading at the end of the backup run, in which case the application will continue to download past the timeout period. If it’s showing “Checking and finishing any potential active exports…” toward the end of the run, it’s waiting for active downloads to finish.

 

Backup ZIP file won’t extract

7-Zip is highly recommended for extracting backups. Depending on the sort option you select and the directory chosen on your local machine, you may end up with long path names that the default Windows extract utility doesn’t like. If you get an error when trying to extract an export, you can either use 7-Zip or move the FGDB zip file (or your entire backup) to a shorter path location and extract it there.

 

‘TEMP_FOR_EXPORT’ files left in root folder of portal

This can happen if the backup process is interrupted before it finishes. You may delete these, or they will eventually be cleared out on their own during a subsequent backup run (the process checks for the number appended to the end of the title, which is a timestamp, and deletes any temporary exports created more than 72 hours ago).

 

Differential backups grab a few feature services that haven’t had edits

Rarely, the differential backup feature may grab a few extra services which have not actually had feature edits in the past specified number of days. This is because ArcGIS Online considers a settings change (editing, sync, change tracking, etc.) to be an edit to the service.

 

Note that differential backups are possible only for ArcGIS Online. Portal for ArcGIS does not currently expose the required property necessary for differential backups.

 

Feature service backup fails

Rarely, a feature service will have persistent issues backing up. If you experience a service that persistently fails, first try increasing the timeout value on the Options tab (Scheduled Version: Cell B19 in the parameters file). If you still experience issues, let us know and we can make some adjustments to your application or otherwise support you in resolving the issue.

 

Backup process takes a long time

Backup time depends primarily on the composition of your content. If you have large services with lots of attachments, it takes time for ArcGIS Online (or your server in the case of Portal for ArcGIS) to do the processing required to export feature services as file geodatabases. You can always split your backups by tag, group, folder, or user if you wish to trim your backup time (indeed, this is advisable for orgs/portals with a high number of large feature services).

 

[Scheduled Version Only] Application fails to open or start

Startup issues (nothing happens when you double-click, or you get a “Fatal Error” warning) are caused by one of the following:

 

1. Failing to place parameters.xlsx in the same folder as the application

2. Failing to enter your working directory in the Start In field of your Task Manager task

3. Renaming parameters.xlsx

4. Moving cells around in parameters.xlsx

5. Failing to provide required entries in parameters.xlsx

If your application fails to start, verify the above and try again. If you need a fresh parameters.xlsx, you can download one here (if Windows gives you a “Can’t Open in Protected Mode” error, just rename the file and it will open in Protected Mode. Just be sure to change the name back to ‘parameters.xlsx’).

 

[Scheduled Version Only] Email confirmation not received

If you’re not receiving email confirmations despite entering an email address in parameters.xlsx, you may need to whitelist ‘info@civiclens.com’, or check your spam/junk folder. Emails are sent as plain text without attachments.

 

Your machine also needs to be able to access smtp.gmail.com to make the call to send the email. This is sometimes blocked by default when running on a server.

 

​

SCHEDULED BACKUP CONFIGURATION

 

Follow the steps below to configure scheduled backups. If you would like to be walked through this process, contact us at support@civiclens.com.

​

Please pay special attention to the following:

 

1. The parameters file must be in the same folder as the application.

2. The parameters file must have valid entries for all fields marked Required.

3. The parameters file must not have been renamed (it must remain ‘parameters.xlsx’).

4. The directory must be provided in the Start In field as shown in Step 6 below.

 

1. Download the application and the parameters.xlsx file and place them in the same folder. If you don’t have Excel on your machine (for example, on a server), you can use parameters.csv (version 3.9.2 or later only).

 

1. Open parameters.xlsx and enter your parameters. Do not rename parameters.xlsx. If you would like to store your ArcGIS password in Windows Credentials Manager instead of directly in the spreadsheet, see Page 12.

 

1. Double-click the application to start a backup and test your parameters. It’s best to let the backup finish, but you can abort it if you wish by simply closing the console window. Note that if you abort the backup while items are exporting, you may be left with a stray ‘TEMP_FOR_EXPORT’ file geodatabase in your org content (you can just delete it).

 

​

1. Open Task Scheduler and click Action > Create Task:

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

1. On the General tab, provide a name and description, and select your security options. Configuring your security options as shown below is ideal, although your overall machine settings will dictate whether you need to run with highest privileges or under a specific user account.

 

 

 

If you wish to see a console window showing progress while the application runs, you must select “Run only when user is logged on”. Otherwise, the application will run non-interactively (in the background).

 

 

1. Select the Triggers tab, click New, and configure an interval.

 

1. Select the Action tab, click New, then click Browse and select ‘Backup_Utility_3_4_Scheduled.exe’ (or whatever you’ve renamed the Backup Utility exe to. If you rename the exe, don’t use spaces.).

 

1. IMPORTANT: Enter your working directory (the directory where you put the exe and parameters.xlsx) in the ‘Start In’ field:

 

 

 

 

 

 

If the following message appears, it’s because the exe name has spaces. Click Cancel, rename the exe to remove spaces, and restart the task configuration.

 

 

 

 

 

 

 

 

 

 

1. Select your preferences on the Conditions and Settings tabs. On the Settings tab, under “If the task is already running, then the following rule applies", it’s advisable to select “Run a new instance in parallel.”

 

1. Click Task Scheduler Library in the upper left. Your tasks will appear in the top pane in the main Task Manager window. Find your new task, right-click and select Run to test it. The application should start in about 30 seconds. If it fails to start, see the Troubleshooting section above.

 

 

 

 

1. ​

 

Depending on your task configuration (see Step 4), you may or may not see a console window appear during the process. If you do see a console window, it’s best to leave it alone while it’s running. If you do interact with it, note the following:

 

If you click anywhere or select text in the console window, it may pause the process/console window output (this is a Windows feature) until you press the Escape key or right-click anywhere in the window. While the backup is running, do not try to copy text from the console window using CTRL-C, as this is a standard Windows command that may interrupt part of the process.

 

 

Storing ArcGIS Password in Windows Credential Manager - Backup Utility v3.9.2 or later (scheduled version only)

 

You can enter your ArcGIS credentials directly in the parameters.xlsx file, or you can choose to store them in Windows Credential Manager by doing the following:

 

1. Open Windows Credential Manager, Select Windows Credentials, and click ‘Add a generic credential’.

2. Enter your connection URL in the ‘Internet or network address’ field, provide your ArcGIS credentials, and click OK.

 

 

Your ArcGIS password

Must match cell B4

Must match cell B3

 

 

 

 

1. In parameters.xlsx, enter “wcm” (without quotes) for your ArcGIS password. The ArcGIS Username must match the username provided to Windows Credential Manager.