The trick is to use the find command with the -exec or -execdir parameter. I needed to remove all the files named serials_dev.db3 from multiple backup directories. The following did the job:

find . -type f -name "serials_dev.db3" -exec rm -f {} ;

Here are the relevant parts from the find man page:

       -exec command ;
              Execute command; true if 0 status is returned.  All following
              arguments to find are taken to be arguments  to  the  command
              until  an  argument  consisting  of  `;' is encountered.  The
              string `{}' is replaced by the current file name  being  pro‐
              cessed  everywhere it occurs in the arguments to the command,
              not just in arguments where it is alone, as in some  versions
              of  find.   Both  of  these  constructions  might  need to be
              escaped (with a `') or quoted to protect them from expansion
              by  the  shell.  See the EXAMPLES section for examples of the
              use of the -exec option.  The specified command is  run  once
              for each matched file.  The command is executed in the start‐
              ing directory.   There are unavoidable security problems sur‐
              rounding use of the -exec action; you should use the -execdir
              option instead.

       -exec command {} +
              This variant of the -exec action runs the  specified  command
              on  the  selected  files,  but  the  command line is built by
              appending each selected file name at the end; the total  num‐
              ber  of invocations of the command will be much less than the
              number of matched files.  The command line is built  in  much
              the  same  way that xargs builds its command lines.  Only one
              instance of `{}' is allowed within the command.  The  command
              is executed in the starting directory.

       -execdir command ;

       -execdir command {} +
              Like  -exec, but the specified command is run from the subdi‐
              rectory containing the matched file, which  is  not  normally
              the  directory  in  which you started find.  This a much more
              secure method for invoking commands, as it avoids race condi‐
              tions  during  resolution  of the paths to the matched files.
              As with the -exec action, the `+' form of -execdir will build
              a command line to process more than one matched file, but any
              given invocation of command will only list files  that  exist
              in  the  same subdirectory.  If you use this option, you must
              ensure that your $PATH environment variable does  not  refer‐
              ence  `.';  otherwise,  an attacker can run any commands they
              like by leaving an appropriately-named file in a directory in
              which  you  will  run  -execdir.   The same applies to having
              entries in $PATH which are empty or which  are  not  absolute
              directory names.