dbfilediff

This takes care of making commands to upload all files not DataEase data files and to patch all tables owned by the developer. Db file diff will create the scripts used for this by dbappinstallation.

First a list of files from the local app that not are db files are created. A list of exclude files or exclude paths can be given before the list is made. 

Files that are considered DataEase files are a file with extension '.tdf', '.dbm', '.bfb', '.bin', '.rdr', '.ssn', '.lan', '.tbr', '.ini', '.log', '.session' or are a files with extension .iXX where XX are a number as these are used by DataEase as index for a field in ta table. To avoid problems with source control systems and code editors all files and folders starting with "." is also excluded from the list. A list from local and deployed are made. If the files have same size a hash is made for each of these files to see if they are the same. If found only one place the file are marked with OnlyDevelopment or OnlyDeployed. After the hash is done for file that exist in both places with same file size, we get a list of all files needed to copy to deployed. No files are marked for deleting as this is considered a manual task.

            ['Relationships','Development','Replace'],
            ['Users','Independent','Nothing'],
            ['Configuration','Independent','Nothing'],
            ['Custom Functions','Independent','Nothing'],
            ['Application Objects','Development','Replace'],
            ['$DeARDL_Registry$','Development','Replace'],
            ['$$DQLStore$$','Development','Replace'],
            ['WebServer','Development','Replace'],

The list of tables to replace data from the developer or not to touch is hard coded to this. If a table in this list exists with Replace, it is added as replacedata command to the jobs list.

The data structures

To run db app installation jobs we have from 1 to 4 data structures to help us, depending on what to do.

dbappjobs

This is a array of array of string. The structure is [[command,what],[command2,what2,basename]...]

The command will tell what is needed of other data. A full overview of all command at the end of this document but it can be copyfile, installdata, deleteform etc.

What is what to do something with like Address table or $$DQLStore$$ table.

There can in some cases where you do not have a proper dbappdatafile structure be a third string telling the base name to install or replace. This is the name used to look up information in the source to be installed. This can be files in f.ex. dbappfiles like 'ADDRAAA' if we are installing a Address table using installdata for just the file listing.

dbappfiles

This is the list for where to find files. When uploading files to the server, we do not know where to find the files by repo paths, so this is a map of filenames mapped to their actual position to copy from. 

Ex. of a list of files to be used for installing a form and a table

{
  'ADDRAAA.FRM' : 'C:\\TEMP\\UPLOAD\\ADDRAAA.FRM',
  'ADDRAAA.I02' :'C:\\TEMP\\UPLOAD\\ADDRAAA.I02'
}

dbappdatafile

This is data from the the local repo about rdrr record etc. It is created as a dict where the form/table name is the key and the info are another dict.

The info dict needs the following information for a replacedata job that is run on a remote computer, for local replacedata the information are found by opening both repositories both the local and the deployed by using the settings, but you can also supply the date

The definition of the inner dict:

- rdrridx | is the index in local repo for the rdrr record. This is more like information of where it came from. Not needed.

- rdrrrecord | is the rdrr record as found in local repo as a dict

- files | the list of files used in the local repo for this table

- formdata | used by install form to figure out frm name and to create a record in Application Objects. The structure is the same as a record in Application Objects. 

- relationships | used by install or replace relations. I is a array with record definitions as read from the Relationships system table.

'dbappdatafiles' : {
    'Address' : {
        'rdrrrecord' : {'rname': 'Address', 'vflag': 0, 'ctlvalue': 25344, 'changeto': '', 'rfilename': 'ADDRDAAA.DBM', 'notup': 0, 'rdrctl': 14, 'nodtup': 0, 'rdescrip': 802, 'cfm_filename': '', 'recno': 19, 'lrsize': 786},
        'files' : ['ADDRDAAA.DBM', 'ADDRDAAA.TDF','ADDRDAAA.I0B','ADDRDAAA.FRM'],
        'formdata' : {'Current Mode': '1', 'Description': '11/03/20;', 'File Name': 'ADDRDAAA.FRM', 'Object Type': '1', 'Object Name': 'Address', 'Style Name': 'Address'}
    },
}

Ex. of table with data and a corresponding frm file for use by installdata + installform

equaldbappfiles

This is a list of files that we have hashed and found equal. This is usually done on a server to save upload of big files. This information will be used by replacedata, replaceform and copyfile to skip files already there that is the same as the one we want to replace. If you need to make this list after you have compared the files using /dbappdatafilescheck/ you can go trough the file list you got back as datafiles and add all the has second string in the string array as "Nothing"

The commands in a job

The commands in dbjobs comes as a array of arrays where the first in the inner array describes the job type.

deletedata Mark the table as deleted in rdrrXaaa.rdr and remove all the files.
deleteform Remove the reference to frm in Application Object and remove the frm file.
filecopy This is the job created for changed or new files. It simply list the file to copy and it will be copied into the application with same name and folder.
-arg1 | filename | the file name is used for both from and to. The from path can either can be looked up in the dbappfiles for full path or created from repopath + filename. The to path is created drepopath + filename
installdata This is a job that add a new table with data to the repository
- arg1 | tablename | the table with data will be installed
installform This adds a entry in Application Objects with a reference to the frm file used for the form. The entry will copy all data from the Application Objects record except for the file name that we have to test if exists. If it exists, we will change the last 3 characters in the file name until we find one that is not used. Rules to use is start on AAA, then AAB till AAZ, then ABA then ABB etc. till we find one.
- arg1 | formname | name of the form to install. It the same as "Object Name" with "Object Type"=1 in "Application Objects"
installrelations Install relactionships for a table.
- arg1 | tablename
installorreplacedata Check if table exists and select installdata if not and replacedata if it do
installorreplaceform Check if form and underlying table exists and install with data if not and replace if with data if it do. This function looks at the Application Objects with "Object Name"=forname and "Object Type"=1 to determine if the form exists and then do a installform or replace form.
To install the data as well you need to use an installorreplace data with what is found in "Style Name". This is the name of the table used for the data.
installorreplacerelations Install all new relationships and update all existing
- arg1 | table name
replacedata This is a job where we supply files for a table that is owned by the developer and it is patched into the deployed version as is with developers data.
- arg1 | tablename | the table data that will be replaced
replaceform This takes a frm file and a formdef like  a record from Application Objects and replace the file defined on target with the one given in Application Objects File Name.
- arg1 | formname | the same as "Object Name" with "Object Type"=1 in "Application Objects"
replacerelations Update existing relationships.
- arg1 | tablename

Supported job types

replacedata

This is what you need for doing a replacedata job. 

When you have a table that you want to do a replacedata to, you first need to do a dbappexpandfilelist by get the rdrr record for the table in local rdrr file. 

Then you must create a list of all the data files the table have locally.This is .DBM, .IXX, .bin and .bfb.

Next you need to run a compare of local and server version of the files by doing a /dbappdatafilescheck/ of local and server version of the files to check if they are equal. This is done by making a datafiles list containing [filename, size, hash] and send this to server. The server responds by a list back with a array of strings [filename, result] where you have the filename and either "Copy" or "Nothing" for the result. Nothing means they are equal. 

All files marked as "Copy" must then be uploaded to the server using /uploadfile/dname/ that takes the filename to be stored on the server as a query paramters ?filename=filenametouse and then posted to the server. The file will be stored in a temp direcotory named the same as the dname used for the application.

Next you can run the job by post to /dbappinstall/dname/ whit a json object of the job description. It must contain jobs, dbappfiles, dbappdatafiles and equaldbappfiles.

{
  "dbappjobs": [
    ["replacedata","Application Objects"],
    ["replacedata","WebServer"]
  ],
  "dbappfiles": {
    "APPLDAAA.DBM": "c:\\users\\bjelg\\appdata\\local\\temp\\dg3deploysetup\\APPLDAAA.DBM",
    "APPLDAAA.I02": "c:\\users\\bjelg\\appdata\\local\\temp\\dg3deploysetup\\APPLDAAA.I02",
    "APPLDAAA.I04": "c:\\users\\bjelg\\appdata\\local\\temp\\dg3deploysetup\\APPLDAAA.I04",
    "APPLDAAA.I06": "c:\\users\\bjelg\\appdata\\local\\temp\\dg3deploysetup\\APPLDAAA.I06",
    "WEBSDAAB.DBM": "c:\\users\\bjelg\\appdata\\local\\temp\\dg3deploysetup\\WEBSDAAB.DBM",
    "WEBSDAAB.I02": "c:\\users\\bjelg\\appdata\\local\\temp\\dg3deploysetup\\WEBSDAAB.I02",
    "WEBSDAAB.bfb": "c:\\users\\bjelg\\appdata\\local\\temp\\dg3deploysetup\\WEBSDAAB.bfb",
    "WEBSDAAB.bin": "c:\\users\\bjelg\\appdata\\local\\temp\\dg3deploysetup\\WEBSDAAB.bin"
  },
  "dbappdatafiles": {
    "Application Objects": {
      "rdrridx": 4,
      "rdrrrecord": {"status": "OK","rname": "Application Objects","vflag": 0,"ctlvalue": 10814,"changeto": "","rfilename": "APPLDAAA.DBM","notup": 6,"rdrctl": 14,"nodtup": 0,"rdescrip": 802,"cfm_filename": "","recno": 4,"lrsize": 213},
      "files": ["APPLDAAA.DBM","APPLDAAA.I02","APPLDAAA.I04","APPLDAAA.I06"]
    },
    "WebServer": {
      "rdrridx": 21,
      "rdrrrecord": {"status": "OK","rname": "WebServer","vflag": 0,"ctlvalue": 17668,"changeto": "","rfilename": "WEBSDAAB.DBM","notup": 9,"rdrctl": 14,"nodtup": 0,"rdescrip": 802,"cfm_filename": "","recno": 21,"lrsize": 411},
      "files": ["WEBSDAAB.DBM","WEBSDAAB.I02","WEBSDAAB.bfb","WEBSDAAB.bin"]
    }
  },
  "equaldbappfiles": []
}

Example of a job list created for replacedata where all files needs to be replaced and none are equal to the local one. All files are already uploaded to the temp area as seen in dbappfiles.