Jump back to the top page Jump back to the top page
 

 Home
 What is XXCOPY ?
 Products
 Downloads
 F A Q 
 Order
 Support
 About Us (Pixelab) 
On-Line Manual 
XXTB #001 
XXTB #002 
XXTB #003 
XXTB #004 
XXTB #005 
             . . .  


XXCOPY
Version
3.33.3


Released
2016-10-28
©2016 Copyright
Pixelab

 All rights reserved  
    [ Table of Contents ] [ Show as Detached ] [ >> ]

    XXCOPY TECHNICAL BULLETIN #005

    From:    Kan Yabumoto           tech@xxcopy.com
    To:      XXCOPY user
    Subject: The Exclusion specifier in XXCOPY
    Date:    2010-12-07  (revised)
    ===============================================================================
    
    
    Much of the mostly hidden power of XXCOPY lies in the exclusion
    mechanism.  Starting with v.2.97.7 (2009-01-13), the exclusion
    specifier is fully compatible with the "Wild-Wildcard" feature.
    That is, you may use wildcard characters (* and ?) anywhere in
    the exclusion specifier in any number of times.  (Earlier 
    versions of XXCOPY had complicated rules on the wildcard usage).
    
    
    XXCOPY Exclusion switch syntax
    
      /X<xspec>       excludes the file or directory item given by
                      <xspec> which is an exclusion specifier.
                      If the specifier contains an embedded space,
                      the specifier must be surrounded by a pair
                      of double-quotes (").
    
      /EX<xfile>      specifies a text file whose name is <xfile>
                      which contains a list of <xspec> separated by space.
    
    
    
      You may specify as many exclusion specifiers as you like.
    
    
    Some examples of the /X switch
    
      The exclusion parameter must be specified immeidately after /X
      (without a space).
    
        /Xc:\mydir\myfile.txt // specifies just a single file
        /X*.tmp               // all files that end with ".tmp"
        /Xabc*                // all files that start with "abc"
    
      To improve readability, we suggest inserting a colon after X.
      Here are more examples:
    
        /X:mydir\             // the entire directory, "mydir" in the source
        /X:mydir\*\*          // same as /Xmydir\ which is a shortcut
        /X:mydir\*\*.tmp      // inside mydir, all files matching "*.tmp"
        /X:my*xyz\*\abc*.c    // in directories that match my*xyz, all "abc*.c"
        /X:*\cache\           // multiple-level subdirectories
        /X:*\cache\*\*        // same as above with a trailing backslash
        /X:*\cach?\*\*        // multiple-level subdir spec may have wildcards
        /X:"\Program Files\"  // use quotes (") with a pattern with a space
        /X:C:\windows\        // it's OK. Two colons with a full path name
    
      Here, you may see the glimpse of the powerful syntax in the exclusion
      specifier.  The first example seems the most straight forward.  The
      fourth example which ends with a backslash is a shorthand of for the
      common case of excluding a directory (it abbreviates "*\*" which follows).
      Therefore, all of the above examples except the first one contain
      or imply at least one wildcard specifier.  The last example includes
      one asterisks in each of the three parts.
    
      Don't worry about the complexity yet.  At least the first example shows
      a case which you can use it immediately without any further reading.
      Yes, if you have energy to list all of the files you want to exclude,
      you may painstakingly list all of such files by giving the full
      file specification of each file.  Since you will soon run out of the
      command line space, you will probably want to set up a text file using
      the /EX switch.
    
    
      E.g.,  /EXmyexcl.lst
    
       and myexcl.lst  contains the following specifiers:
    
         // this is a comment line
         c:\win386.swp               // comment may start like this
         c:\autoexec.bat  myfile.tmp // one line may have multiple items
         "c:\program files"          // use quotes (") for embedded space
         mydir\myfile.txt            // pathspec relative to the source dir
         yourdir\                    // entire yourdir\*\*
    
    
    Characteristics of exclusion specifier (a short explanation):
    
        1.  The Wild-Wildcard feature fully supported for exclusion.
        2.  A pattern for direcotires always ends with a backslash.
        3.  A pattern for files has no trailing backslash.
        4.  A relative specifier is relative to the source base directory.
    
    
    Syntactic rules of exclusion specifier:
    
        1.  Add the exclusion item immediately (without space) after /X.
    
             /Xmyfile.txt          // exclude "myfile.txt" anywhere
    
        2.  You may insert an optional colon (:) between /X and the rest.
    
             /X:xyz.doc            // the colon improves the readability
    
        3.  The trailing backslash distinguishes directory from file.
    
             /X:Windows\          // specifies the "Windows" directory
             /X:pagefile.sys      // specifies the "pagefile.sys" file
    
        4.  Use a pair of quotes (") for an exclusion item with embeded space.
    
             /X:"\Program Files\"  // the space would split the string
    
        5.  Absolute exclusion specifiers start with a backslash
            (with an optional drive letter or UNC pattern).
    
             /X:c:\mydir\myfile.txt     // an absolute pattern
             /X:\mydir\myrecord.doc     // you may omit the drive letter
             /X:\\john\c\windows\       // UNC always starts with \\
    
        6.  Relative exclusion specifiers (without leading backslash)
            are relative to the source base specifier.
    
             /X:dir_abc\myfile.txt      // relative to the source base dir
    
        7.  A bare exclusion item (without any backslash) is treated
            as a file name pattern that applies to all directories.
    
             /X:desktop.ini             // equivalent to /X:*\desktop.ini
             /X:*.mp3                   // equivalent to /X:*\*.mp3
    
            This feature allows for a filename pattern to apply for all
            directories.  On the other hand, you may want to specify a
            file pattern that explicitly apply only to the files in the
            source base directory.  In such a case, use a ".\" prefix.
    
             /X:.\*.tmp                 // for files in the top directory
    
        8.  A bare directory pattern (the only backslash is at the end)
            is treated a pattern that applies only to the source base
            directory, unlike the bare file name case exlained above.
    
             /X:dir_xyz\                // applies only the top directory
             /X:.\dir_xyz\              // the same as the one above
    
    
    The Wild-Wildcard feature in exclusion specifier:
    
        Starting with v.2.97.7 (2009-01-01), you may use wildcard character
        anywhere in an exclusion specifier for any number of times.
    
           Astrisk (*) denotes a string of zero or more of any character.
    
           Question Mark (?) denotes exactly one character.
    
    
             /X:*.mp3      // any file that ends with ".mp3"
             /X:abc*\      // any directory that starts with "abc"
             /X:xyz???     // starts with xyx followed with 3 more chars
             /X:*a*        // a filename that has at least one letter "a"
    
    
        The Special "wild subdirectory"  (..\*\..) notation:
    
            The \*\ (an asterisk sandwiched by backslashes) sequence
            denotes a pathname pattern with zero or more of directories.
    
             /X:c:\mydir\*\abc\    excludes the following directories
    
                c:\mydir\abc\
                c:\mydir\xyz\abc\
                c:\mydir\klm\xyz\abc\
    
            Such a pattern does not always match the first one.
    
             /X:c:\mydir\*\a\b\    excludes the following directories
    
                c:\mydir\a\b\           // the first instance
                c:\mydir\a\xyz\a\b\     // 1st \a\ does not but 2nd one does
    
    
     Syntax rule for the Exclusion List File (/EX).
    
        An "Exclusion List File" specified in the /EX switch is a plain
        text file which contains a list of exclusion specifiers.
        You may list as many exclusion specifiers in one line.
        Exclusion specifiers are separated by one or more blank, tab,
        and/or newline character.  An exclusion specifier cannot be
        broken into two or more lines.  When a space character is
        embedded, the exclusion specifier must be surrounded by a
        pair of double-quotes (").  A line may contain a comment field
        which will be ignored by XXCOPY.  A comment field starts with
        two consecutive slashes (//) or two consecutive colons (::)
        and ends at the end of the line.  You may also use another
        comment style (starts with /* and ends with */ that can span
        over multiple lines).
    
    
    
    A few examples of common exclusion specifiers.
    
        The directory template may have the following four variations
        in the ending.
    
          dir_pattern\         // full directory
          dir_pattern\*\*      // full directory (same as above)
          dir_pattern\*        // files in the directory (one-level)
          dir_pattern\?*\      // all subdirectories but not files
    
          Even though the second example is to specify files (as opposed
          to directories) without the mandatory trailing backslash,
          it behaves exactly like the first one because it covers
          all files in all subdirectories which is equivalent to the
          whole directory contents that is the directory itself.
    
    
        Examples:
    
            /X:dirtmpl\*\*     // excludes all matching directories in the
                               // base (src) directory and its contents
            /X:dirtmpl\        // same as above (the triling backslash
                               // denotes everything inside the directory)
            /X:.\dirtmpl\      // in the case of the directory template,
                               // it applies to one directory relative to
                               // the base (src) directory (1-Level)
            /x:*\dirtmp\       // you may make a directory template apply
                               // to many instances (Multi-Level)
    
    
            /x:c:\windows\*    // specifies all the files in the first
                               // level of the c:\Windows directory such
                               // as, EXPLORER.EXE, WIN.INI,
    
            /x:c:\windows\?*\  // this does not includes the first level
                               // files but all subdirectories in it such
                               // as \WINDOWS\SYSTEM\  \WINDOWS\DESKTOP\, etc.
    
    
         Note that the following two are distinct:
    
             /Xdir_spec\*     // one layer only (subdirectories not excluded)
             /Xdir_spec\*\*   // the entire dir_spec directory is excluded
    
    
         Beware of a few pitfalls:
    
             xxcopy mydir\  x:\mydst\  /S /X:mydir\a*\
    
                The exclusion item specifies a pattern (directory starting with
                letter a) as a relative path.  As you explained above, it is
                relative to the source base directory.  Therefore, this command
                excludes directories that matches  mydir\mydir\a*\  not
                the subdirectories immediately in the mydir\.  Rather, the
                exclusion specifier stipulate mydir inside the source base
                directory which happens to spell the same.
    
             xxcopy \src\  \dst\      /S /X:*.tmp
    
                The exclusion specifier is equivalent to /X:*\*.tmp that applies
                to all subdirectories.
    
             xxcopy "C:\Program Files\"  \dst\   /S /X:Microsoft*\
    
                The exclusion specifier deos not implicitly applies to all
                subdirectories as the filename template does.  In this case, only
                directories whose name starts with Microsoft immediately inside
                the C:\Program Files\ directory, but not at other levels.
    
    
    The optimization of exclusion matching.
    
        In a very large scale backup operation, an XXCOPY job may encompass
        an entire volume as the source directory (such as c:\*).  To make
        the matters worse, the more files the source directory contains,
        the more the need for the exclusion specifiers grows.  Therefore,
        it is entirely possible that the entire C: drive may contain
        70,000 files and the total number of exclusion items the user specify
        in the exclusion list file with the /EX switch may contain literally
        hundreds of various exclusion specifiers.  If we were to test every
        file against this very large number of exclusion list, the combination
        will easily reach tens of millions which would slow down the entire
        backup process.  Therefore, XXCOPY performs preprocessing steps
        to analyze the set of exclusion specifiers.  First by classifying
        them into the five classes, some redundant exclusion specifiers can
        be removed.
    
    
    Debug feature
    
        Because a liberal use of wildcard characters in exclusion feature
        could become fairly complex, sometimes the user may want to test
        if a given set of exclusion items really achieve the intended
        result.
    
        The /oX switch generates a list of exclusion specifiers after
        XXCOPY's initial exclusion parameter optimization step.  The list
        is organized by the internal classification of the wildcard usage.
        It is most effective in combination with the /DEBUG function.
        (This output is mostly for Pixelab's support activities.)
    
        The /ZLX function provides a special test by listing all the files
        and directories that would be excluded without actually making any
        copy or delete operation.  It suppresses the display of regular
        (selected) files and generates a list of files and directories that
        would be excluded by the /X and /EX switches.  It is analogous to
        the /L function that lists the files that would be selected by
        a given set of command arguments.  
    
    
    Automatically excluded files.
    
        Since the few output files (e.g., the error log files) which are
        generated by the XXCOPY program itself could not be successfully
        included in the current copying job if any of them happens to be
        in the source directory (or its subdirectories), those files will
        be automatically excluded.
    
    
    Note on earlier (pre v.2.97.7) versions:
    
        In earlier versions of XXCOPY (prior to v.2.97.7), the exclusion
        feature was the hardest one to fully master by XXCOPY's users.
        That was because the exclusion specifier in earlier versions
        did not support the "Wild-Wildcard" capablitity to the fullest
        extent.  Rather, XXCOPY provided a total of 11 classes of wildcard
        usages with a complex set of syntactic rules.  So, the many users
        had to stick with a few simplest ways to add wildcard characters
        for the exclusion items.  It was simply very difficult to implement
        an efficent algorithm to support such a feature.
    
        In the old (pre-v.2.97.7) versions, the exclusion feature defined
        a special syntactic rule as follows:
    
              dir_pattern\?\*
    
        When an exclusion item with the \?\* ending, it excludes the
        subdirectories of the directories that matches the pattern.  The
        above exclusion pattern can be written as
    
             dir_pattern\?*\
    
        which does not rely on the special syntactic rule (which is now
        being abolished).
    
    
    Use of the /GX switch:
    
        In order to support the old syntax for the \?\* ending in the
        exclusion specifier, we have added the /GX switch with which
        a command line with the older exclusion specifier is treated
        in the old way.
    
        However, we urge all users to replace all usages of \?\* ending
        with \?*\ that excludes exactly the same group of subdirectories.
    
    
    
    
    [ Table of Contents ] [ Show as Detached ] [ >> ]