Small tip: /Fully/ backup your (psql) CMS library

Hi gang!

As we all probably know all your contents basically get stored in several ways. First is the content library; the actual (physical) location where Daz Studio ("DS") stores all its contents. And it's also the place where you can add downloaded contents yourself (though using your personal library might be safer). This location is easily accessible using the Content Library pane within DS (when in doubt check out the Windows menu: Windows => Panes (Tabs) => Content Library).

Next there's another storage area: the CMS, powered by PostgreSQL. This is actually the thing which powers the smart content pane, the area where you can easily pick out all the assets you want to add to your scenery and which will automatically filter the displayed items based on your selection.

Now... DS allows for database maintenance which also allows you to export your user data. Its easy: right click on the Content Library tab and select "Content DB Maintenance...", this is your main area for most common maintenance issues. From re-scanning your contents, marking all as seen right down to consolidating, condensing and... Exporting your data.

But here's my reason for all this: it only exports user data (according to the documentation), but what about the rest? When I back something up I want a full backup. Fortunately for us PostgreSQL allows for just that, and its easy too:

  • Fire up DS or the DIM but don't do anything in particular. This will also start the CMS (PostgreSQL).
  • Find your DS program folder, the main folder should also contain something called: "PostgreSQL CMS" which is what we're after.
  • Open a commandline (cmd.exe on Windows) and navigate to this folder, specifcally the bin folder underneath it.
    • On Windows: Win-R => Enter "cmd" (don't include "") to start it => use 'cd' to go to the program folder. In my case I'd use: cd "\program files\daz 3d\postgresql cms\bin". Note that you can use TAB for command completion.
  • Enter this: pg_dump -f ..\daz-cms.sql -d Content -p 17237 -U dzcms -w
    • This will create a (text) file called daz-cms.sql in the directory above it which will contain all of your CMS contents.

Example and proof of concept:

Now for step 2...

Restoring your data

Warning, warning, warning: Do NOT use this option casually, only resort to this method when the other options don't work for you. DS provides several options on its own (like the option to re-import all your meta data) and those should be tried before resorting to all this. The reason I mention this anyway is two folded: what good is a backup without a means to restore? And because I'm not too sure how well DS will cope with any manually added meta data (like manually adding contents to the smart content pane).

  • Use all the steps above to end up with a running PostgreSQL and naviate to the bin directory using the commandline.
  • Then use this command (this assumes the previously created text file to be in the above directory): pg_restore -f ..\daz-cms.sql -d Content -p 17237 -U dzcms -w.
  • Close DS (or the DIM) without making any chances and wait a moment before restarting it.

Once again I need to stress out that you shouldn't use this method casually, but it can be a full proof way of making sure that all your CMS data is kept safe.

Hope this can benefit someone.

 

 

 

 

 

 

Comments

  • andya_b341b7c5f5andya_b341b7c5f5 Posts: 694
    edited February 2016
    ShelLuser said:

    Hi gang!

    As we all probably know all your contents basically get stored in several ways. First is the content library; the actual (physical) location where Daz Studio ("DS") stores all its contents. And it's also the place where you can add downloaded contents yourself (though using your personal library might be safer). This location is easily accessible using the Content Library pane within DS (when in doubt check out the Windows menu: Windows => Panes (Tabs) => Content Library).

    Next there's another storage area: the CMS, powered by PostgreSQL. This is actually the thing which powers the smart content pane, the area where you can easily pick out all the assets you want to add to your scenery and which will automatically filter the displayed items based on your selection.

    Now... DS allows for database maintenance which also allows you to export your user data. Its easy: right click on the Content Library tab and select "Content DB Maintenance...", this is your main area for most common maintenance issues. From re-scanning your contents, marking all as seen right down to consolidating, condensing and... Exporting your data.

    But here's my reason for all this: it only exports user data (according to the documentation), but what about the rest? When I back something up I want a full backup. Fortunately for us PostgreSQL allows for just that, and its easy too:

    • Fire up DS or the DIM but don't do anything in particular. This will also start the CMS (PostgreSQL).
    • Find your DS program folder, the main folder should also contain something called: "PostgreSQL CMS" which is what we're after.
    • Open a commandline (cmd.exe on Windows) and navigate to this folder, specifcally the bin folder underneath it.
      • On Windows: Win-R => Enter "cmd" (don't include "") to start it => use 'cd' to go to the program folder. In my case I'd use: cd "\program files\daz 3d\postgresql cms\bin". Note that you can use TAB for command completion.
    • Enter this: pg_dump -f ..\daz-cms.sql -d Content -p 17237 -U dzcms -w
      • This will create a (text) file called daz-cms.sql in the directory above it which will contain all of your CMS contents.

    Example and proof of concept:

    Now for step 2...

    Restoring your data

    Warning, warning, warning: Do NOT use this option casually, only resort to this method when the other options don't work for you. DS provides several options on its own (like the option to re-import all your meta data) and those should be tried before resorting to all this. The reason I mention this anyway is two folded: what good is a backup without a means to restore? And because I'm not too sure how well DS will cope with any manually added meta data (like manually adding contents to the smart content pane).

    • Use all the steps above to end up with a running PostgreSQL and naviate to the bin directory using the commandline.
    • Then use this command (this assumes the previously created text file to be in the above directory): pg_restore -f ..\daz-cms.sql -d Content -p 17237 -U dzcms -w.
    • Close DS (or the DIM) without making any chances and wait a moment before restarting it.

    Once again I need to stress out that you shouldn't use this method casually, but it can be a full proof way of making sure that all your CMS data is kept safe.

    Hope this can benefit someone.

     

     

     

     

     

     

    Good information. 

    However, does it not go to show why using an enterprise grade database just to keep track of simple product metadata within a single application is an odd design choice?  One reason among many being that a database like Postgres needs a certain level of knowledge to manage and administer it properly, as your post demnstrates, that the average DS user should not be expected to need to acquire.  And if they manage to make a full backup, the next question is: how/where are they going to test that nice backup they made?.  I know I don't have a spare machine to test them on (though virtual would do), and if you don't know you can actually restore from a backup, it is pretty much worthless.

    Post edited by andya_b341b7c5f5 on
  • fixmypcmikefixmypcmike Posts: 19,619

    Why would you need to do this, since non-UserData already has backups in Runtime:Support (for DIM or manual installs) and Data/Cloud/Meta for Connect installs?

  • Why would you need to do this, since non-UserData already has backups in Runtime:Support (for DIM or manual installs) and Data/Cloud/Meta for Connect installs?

    I'm afraid I'm not familiar with the files stored in these locations.  Am I right in thinking it is the files downloaded by DIM and Daz Connect?  If so, they are not a backup of the PostgreSQL database, though using them to re-install products may be a recovery plan some Daz users are happy with. 

    But the answer to your question why you would do this is essentially as stated in the PostgreSQL docs: "As with everything that contains valuable data, PostgreSQL databases should be backed up regularly." (http://www.postgresql.org/docs/9.1/static/backup.html).  I suppose it may depend how 'valuable' an individual thinks product metadata is as to whether database backups seem like too much bother.  Personally, I don't have PostgreSQL installed, and I just backup the whole disk as an image and be done with it.

    The PostgreSQL docs are well worth a read - an example of how good documentation can be for software that genuinely costs nothing.

  • Why would you need to do this, since non-UserData already has backups in Runtime:Support (for DIM or manual installs) and Data/Cloud/Meta for Connect installs?

    Thank-you.... I was wondering where the "UserData1" was located... It was ruining my smart-content. (That is the database backup, not JUST userdata.)

    Stupid use for it... It should be only userdata, and backups should be backups/exports.

  • namffuaknamffuak Posts: 4,238

    Why would you need to do this, since non-UserData already has backups in Runtime:Support (for DIM or manual installs) and Data/Cloud/Meta for Connect installs?

    I'm afraid I'm not familiar with the files stored in these locations.  Am I right in thinking it is the files downloaded by DIM and Daz Connect?  If so, they are not a backup of the PostgreSQL database, though using them to re-install products may be a recovery plan some Daz users are happy with. 

    But the answer to your question why you would do this is essentially as stated in the PostgreSQL docs: "As with everything that contains valuable data, PostgreSQL databases should be backed up regularly." (http://www.postgresql.org/docs/9.1/static/backup.html).  I suppose it may depend how 'valuable' an individual thinks product metadata is as to whether database backups seem like too much bother.  Personally, I don't have PostgreSQL installed, and I just backup the whole disk as an image and be done with it.

    The PostgreSQL docs are well worth a read - an example of how good documentation can be for software that genuinely costs nothing.

    The method provided by DAZ works quite well. In Content DB Maintenance (from the Options menu of either the Content Library or Smart Content tabs)  select 'Export User Data'; this creates one or more files in the runtime/support directory at the top of the list of DAZ content directories.

    To re-build the dataabase you can either select 'Reset Database' from the Content DB Maintenance menu, OR (my preferred, the nuclear option) stop the  CMS (Options of the Content Library tab) and then blow away the 'Content Cluster' directory with all the database files in it. Then Start CMS - and Studio bilds the empty database - followed by 'Reimport Metadata'.

  • srieschsriesch Posts: 4,243
    edited October 2018

    As I have recently been having problems with "export user data", I thought I'd give this a try.  A few points.  First, it appears you need to right-click on cmd <dot> exe and select the "run as administrator" option.

    Also, when restoring, the supplied line does not work.  you can't use the -f option (which is apparently for an ouptut file) with pg_restore.  I tried modifying it to put the filename at the end with no -f, and that gave me a message that because the input file appears to be in text format to please use psql.  I then tried the original command but replacing pg_restore with psql, and it started doing something but was generating a lot of errors that things already exist.  It appears that even if you delete the database it gets rebuilt when you run DS, but PostgreSQL appears to need to be running for this to work.  Does anybody know the best way to simulate doing that without actually running DS, or if the errors can actually be ignored and won't prevent anything from working that isn't already there?

    (side note, the forum software will not let you type cmd <dot> exe with a '.'.)

    Post edited by sriesch on
  • srieschsriesch Posts: 4,243

    Any idea what side effects there might be if one tried to do things like drop and re-create tables while DS was running?  Obviously not desirable user behavior, but it may be that or nothing.

Sign In or Register to comment.