SEO

Hi!

Today I realized that some of our scheduled backups were silently
ignored. Here are some details:

In the DP schedule files, you can specify "-at HH:MM" for the time when to
execute the specification. First surprise is that Data Protector reports a
"syntax error" if you specify "MM" that is different from "00", "15", "30",
"45". That is you can only specify multiples of 15 minutes.

In our complex scenario with many hosts and devices, I wrote a backup planner
that reads high-level specifications and the schedules DP backups
(i.e. creates the schedule files, logs, statistics, iCalendar, estimate of
media usage, data protection validation, etc.), considering which
specification uses which resorces, the number of available licenses, holidays,
weekends, etc.

Now that some incremental backups just take 4 minutes or so, the created
schedules would be quite tight. For example:
17:00 A
17:00 B
17:10 A
17:10 B
17:30 C
17:30 D
17:40 B
17:40 D

A to D are backup specifications and the first occurrences are incremental
backups, followed by levelled backups.

As DP cannot schedule at 17:10, the backup was scheduled at 17:00 also. The
expectation was that it would be queued until the first one is finished.
However, as it seems, DP just silently ignored the second occurrence of A, B,
C, and D.

Can anybody explain this arbitrary restriction to muliples of 15 minutes?
Cron can do a better job for 20 years now.

The solution seems to be to specify a minimum duration of 15 minutes per
backup session, even though the backup just needs a few minutes. This will
fragment the time space a lot, being unable to fill the gaps. Also this will
minimize device usage as there are unnecessary breaks between backups. My
scheduler could handle free time slots down to one second, but it does not
make any sense with DP.

BTW: How do you guys keep track when which backups are scheduled to avoid
queueing conflicts caused by scheduling and device usage?

Regards,
Ulrich

"Ulrich Windl" <[email protected]> wrote in message
news:[email protected]...
> Hi!
>
> Today I realized that some of our scheduled backups were silently
> ignored. Here are some details:
>
> In the DP schedule files, you can specify "-at HH:MM" for the time when to
> execute the specification. First surprise is that Data Protector reports a
> "syntax error" if you specify "MM" that is different from "00", "15",
> "30",
> "45". That is you can only specify multiples of 15 minutes.
>
> In our complex scenario with many hosts and devices, I wrote a backup
> planner
> that reads high-level specifications and the schedules DP backups
> (i.e. creates the schedule files, logs, statistics, iCalendar, estimate of
> media usage, data protection validation, etc.), considering which
> specification uses which resorces, the number of available licenses,
> holidays,
> weekends, etc.
>
> Now that some incremental backups just take 4 minutes or so, the created
> schedules would be quite tight. For example:
> 17:00 A
> 17:00 B
> 17:10 A
> 17:10 B
> 17:30 C
> 17:30 D
> 17:40 B
> 17:40 D
>
> A to D are backup specifications and the first occurrences are incremental
> backups, followed by levelled backups.
>
> As DP cannot schedule at 17:10, the backup was scheduled at 17:00 also.
> The
> expectation was that it would be queued until the first one is finished.
> However, as it seems, DP just silently ignored the second occurrence of A,
> B,
> C, and D.
>

Using the standard DP interface, can you actually schedule the one
specification to run twice at the same time? I have not tried, but assume
the GUI would "merge" both of them into one.

> Can anybody explain this arbitrary restriction to muliples of 15 minutes?
> Cron can do a better job for 20 years now.


The omnitrig process is what actually schedules the sessions, and it is only
run every 15minutes (from cron). I suspect this has been hardcoded in
somewhere.

You can however run a session "manually" from the command line ... or from
cron. So you can just ignore DataProtector's scheduling stupidity, and
schedule them directly from cron. Not as pretty, but if you have already
written a scheduler, formatting it's output for cron (vs DP) should not be
too difficult. Look at "omnib -datalist specname -no_monitor"

>
> The solution seems to be to specify a minimum duration of 15 minutes per
> backup session, even though the backup just needs a few minutes. This will
> fragment the time space a lot, being unable to fill the gaps. Also this
> will
> minimize device usage as there are unnecessary breaks between backups. My
> scheduler could handle free time slots down to one second, but it does not
> make any sense with DP.

Be aware that you will probably hit another quirk with DP queuing. When you
have multiple sessions queued (for a device or licence) DP does not respect
queue order. Once the device (or licence) becomes available there does not
seem to be any rational (except for Murphy's Law) as to which session will
proceed first.

>
> BTW: How do you guys keep track when which backups are scheduled to avoid
> queueing conflicts caused by scheduling and device usage?
>

Thankfully (for me) I have a small number of long (full backup) sessions, so
scheduling is not too difficult.

> Regards,
> Ulrich



--
Posted via a free Usenet account from http://www.teranews.com

"Kilgaard" <[email protected]> writes:

> "Ulrich Windl" <[email protected]> wrote in message
> news:[email protected]...
>> Hi!
>>
>> Today I realized that some of our scheduled backups were silently
>> ignored. Here are some details:
>>
>> In the DP schedule files, you can specify "-at HH:MM" for the time when to
>> execute the specification. First surprise is that Data Protector reports a
>> "syntax error" if you specify "MM" that is different from "00", "15",
>> "30",
>> "45". That is you can only specify multiples of 15 minutes.
>>
>> In our complex scenario with many hosts and devices, I wrote a backup
>> planner
>> that reads high-level specifications and the schedules DP backups
>> (i.e. creates the schedule files, logs, statistics, iCalendar, estimate of
>> media usage, data protection validation, etc.), considering which
>> specification uses which resorces, the number of available licenses,
>> holidays,
>> weekends, etc.
>>
>> Now that some incremental backups just take 4 minutes or so, the created
>> schedules would be quite tight. For example:
>> 17:00 A
>> 17:00 B
>> 17:10 A
>> 17:10 B
>> 17:30 C
>> 17:30 D
>> 17:40 B
>> 17:40 D
>>
>> A to D are backup specifications and the first occurrences are incremental
>> backups, followed by levelled backups.
>>
>> As DP cannot schedule at 17:10, the backup was scheduled at 17:00 also.
>> The
>> expectation was that it would be queued until the first one is finished.
>> However, as it seems, DP just silently ignored the second occurrence of A,
>> B,
>> C, and D.
>>
>
> Using the standard DP interface, can you actually schedule the one
> specification to run twice at the same time? I have not tried, but assume
> the GUI would "merge" both of them into one.

They (17:00 and 17:10) are not actually the same: The differ in backup level
and data protection.

>
>> Can anybody explain this arbitrary restriction to muliples of 15 minutes?
>> Cron can do a better job for 20 years now.
>
>
> The omnitrig process is what actually schedules the sessions, and it is only
> run every 15minutes (from cron). I suspect this has been hardcoded in
> somewhere.

I guess they have a fixed-size array to manage the sched