Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Rephrase cycle time to cycle point #1323

Merged
merged 6 commits into from
Aug 8, 2014
Merged
Show file tree
Hide file tree
Changes from 4 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -51,8 +51,8 @@ <h2>suite.rc</h2>
timeout handler = rose suite-hook --mail --shutdown
timeout = 4320 # 3 days
[scheduling]
initial cycle time = 2013060100
final cycle time = 2013060300
initial cycle point = 2013060100
final cycle point = 2013060300
[[special tasks]]
cold-start = startup
[[dependencies]]
Expand Down
4 changes: 2 additions & 2 deletions doc/etc/rose-rug-suite-writing-tutorial/suite.rc.html
Original file line number Diff line number Diff line change
Expand Up @@ -50,8 +50,8 @@ <h2>suite.rc</h2>
timeout handler = rose suite-hook --mail --shutdown
timeout = 4320 # 3 days
[scheduling]
initial cycle time = 2013060100
final cycle time = 2013060300
initial cycle point = 2013060100
final cycle point = 2013060300
[[special tasks]]
cold-start = fcm_make_navigate, write_start_position
[[dependencies]]
Expand Down
6 changes: 3 additions & 3 deletions doc/rose-rug-advanced-tutorials-broadcast.html
Original file line number Diff line number Diff line change
Expand Up @@ -114,8 +114,8 @@ <h3 class="alwayshidden">Task-based Example Suite</h3>
timeout handler = rose suite-hook --mail --shutdown
timeout = 4320 # 3 days
[scheduling]
initial cycle time = 2013010100
final cycle time = 2013010500
initial cycle point = 2013010100
final cycle point = 2013010500
[[special tasks]]
cold-start = startup
[[dependencies]]
Expand Down Expand Up @@ -226,7 +226,7 @@ <h3 class="alwayshidden">Cycle Time Extension</h3>
<p>We should make the suite run a bit longer by increasing <samp>final
cycle time</samp>:</p>
<pre class="prettyprint lang-cylc">
final cycle time = 2013012500
final cycle point = 2013012500
</pre>
</div>

Expand Down
2 changes: 1 addition & 1 deletion doc/rose-rug-advanced-tutorials-clock-triggered.html
Original file line number Diff line number Diff line change
Expand Up @@ -127,7 +127,7 @@ <h3 class="alwayshidden">Example suite.rc</h3>
timeout handler = rose suite-hook --mail --shutdown
timeout = 4320 # 3 days
[scheduling]
initial cycle time = <span id="cylc-time"></span>
initial cycle point = <span id="cylc-time"></span>
[[dependencies]]
[[[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23]]]
graph = bell
Expand Down
8 changes: 4 additions & 4 deletions doc/rose-rug-quiz.html
Original file line number Diff line number Diff line change
Expand Up @@ -739,8 +739,8 @@ <h2 id="questions">Questions</h2>
<samp>scheduling</samp> section:</p>
<pre class="prettyprint lang-cylc">
[scheduling]
initial cycle time = 2014010500
final cycle time = 2014010700
initial cycle point = 2014010500
final cycle point = 2014010700
[[dependencies]]
[[[0, 12]]]
graph = """
Expand Down Expand Up @@ -897,8 +897,8 @@ <h2 id="questions">Questions</h2>
<td class="noborder">
<pre class="prettyprint lang-cylc">
[scheduling]
initial cycle time = 2013060100
final cycle time = 2013060300
initial cycle point = 2013060100
final cycle point = 2013060300
</pre>
</td>
</tr>
Expand Down
4 changes: 2 additions & 2 deletions doc/rose-rug-suite-writing-tutorial.html
Original file line number Diff line number Diff line change
Expand Up @@ -557,8 +557,8 @@ <h3 class="alwayshidden">suite.rc Content</h3>
<p>Replace <samp>[scheduling]</samp> with the following lines:</p>
<pre class="prettyprint lang-cylc">
[scheduling]
initial cycle time = 2013060100
final cycle time = 2013060300
initial cycle point = 2013060100
final cycle point = 2013060300
[[special tasks]]
cold-start = fcm_make_navigate
[[dependencies]]
Expand Down
4 changes: 2 additions & 2 deletions doc/rose-rug-suites.html
Original file line number Diff line number Diff line change
Expand Up @@ -517,8 +517,8 @@ <h3 id="dependencies-cycling">Cycling Dependencies</h3>
<p>We can make this run as a cycling suite, repeating every 12 hours:</p>
<pre class="prettyprint lang-cylc">
[scheduling]
initial cycle time = 2013010500 # 00:00, 5/1/2013
final cycle time = 2013010600 # 00:00, 6/1/2013
initial cycle point = 2013010500 # 00:00, 5/1/2013
final cycle point = 2013010600 # 00:00, 6/1/2013
[[dependencies]]
[[[0, 12]]] # run each day at 00:00 and 12:00
graph = hello_pluto =&gt; hello_eris
Expand Down
4 changes: 2 additions & 2 deletions etc/rose-rug-brief-tour/suite.rc
Original file line number Diff line number Diff line change
Expand Up @@ -7,8 +7,8 @@
timeout = 4320 # 3 days
# End Rose required event hook block
[scheduling]
initial cycle time = 2013010100 # The first cycle of the suite is at 00:00 on Jan 1 2013
final cycle time = 2013010200 # The last cycle of the suite is at 00:00 on Jan 2 2013
initial cycle point = 2013010100 # The first cycle of the suite is at 00:00 on Jan 1 2013
final cycle point = 2013010200 # The last cycle of the suite is at 00:00 on Jan 2 2013
[[special tasks]]
cold-start = fcm_make # This task is only run at the beginning of the suite
[[dependencies]]
Expand Down
4 changes: 2 additions & 2 deletions etc/rose-rug-simple/suite.rc
Original file line number Diff line number Diff line change
Expand Up @@ -6,8 +6,8 @@
timeout=2 # minutes

[scheduling]
initial cycle time=2013010100 # 2013-01-01T00:00:00Z
final cycle time=2013010200 # 2013-01-02T00:00:00Z
initial cycle point=2013010100 # 2013-01-01T00:00:00Z
final cycle point=2013010200 # 2013-01-02T00:00:00Z
[[dependencies]]
[[[0, 6, 12, 18]]]
graph=hello[T-6]=>hello
Expand Down
4 changes: 3 additions & 1 deletion lib/python/rose/date.py
Original file line number Diff line number Diff line change
Expand Up @@ -118,7 +118,9 @@ def date_format(self, print_format, time_point=None):
Otherwise, use current date time.

"""
if "%" in print_format:
if print_format is None:
return str(time_point)
if print_format and "%" in print_format:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

First part of the if isn't necessary here, I think.

try:
return time_point.strftime(print_format)
except ValueError:
Expand Down
14 changes: 11 additions & 3 deletions lib/python/rose/suite_engine_proc.py
Original file line number Diff line number Diff line change
Expand Up @@ -197,15 +197,23 @@ class SuiteStillRunningError(Exception):
"""An exception raised when a suite is still running."""

FMT_HEAD = "Suite \"%(suite_name)s\" may still be running.\n"
FMT_BODY = "Host \"%(host)s\" has %(reason_key)s:\n %(reason_value)s\n"
FMT_TAIL = "Try \"rose suite-shutdown --name=%(suite_name)s\" first?"

FMT_BODY1 = "Host \"%(host)s\" has %(reason_key)s:\n"
FMT_BODY2 = " %(reason_value)s\n"

def __str__(self):
suite_name, reasons = self.args
ret = self.FMT_HEAD % {"suite_name": suite_name}
host = None
reason_key = None
for reason in reasons:
ret += self.FMT_BODY % dict(reason)
reason_dict = dict(reason)
if (reason_dict["host"] != host or
reason_dict["reason_key"] != reason_key):
ret += self.FMT_BODY1 % reason_dict
host = reason_dict["host"]
reason_key = reason_dict["reason_key"]
ret += self.FMT_BODY2 % reason_dict
ret += self.FMT_TAIL % {"suite_name": suite_name}
return ret

Expand Down
28 changes: 17 additions & 11 deletions lib/python/rose/suite_engine_procs/cylc.py
Original file line number Diff line number Diff line change
Expand Up @@ -650,15 +650,18 @@ def is_suite_registered(self, suite_name):
return self.popen.run("cylc", "get-directory", suite_name)[0] == 0

def is_suite_running(self, user_name, suite_name, hosts=None):
"""Return the port file path if it looks like suite is running.
"""Return the reasons if it looks like suite is running.

If pgrep "cylc-{run,restart} suite_name",
return (host, "process", pid).
return [
{
"host": host,
"reason_key": reason_key,
"reason_value": reason_value
},
# ...
]

If port file exists,
return (host, "port-file", "/path/to/.cylc/ports/suite_name").

Return () otherwise.
If not running, return an empty list.

"""
if not hosts:
Expand All @@ -671,11 +674,15 @@ def is_suite_running(self, user_name, suite_name, hosts=None):
"python.*cylc-(run|restart).*\\<" + suite_name + "\\>"]
ret_code, out, _ = self.popen.run(*pgrep)
if ret_code == 0:
proc_reasons = []
for line in out.splitlines():
if suite_name in line.split():
return [{"host": "localhost",
"reason_key": self.REASON_KEY_PROC,
"reason_value": line}]
proc_reasons.append({
"host": "localhost",
"reason_key": self.REASON_KEY_PROC,
"reason_value": line})
if proc_reasons:
return proc_reasons

# remote hosts pgrep and ls port file
host_proc_dict = {}
Expand Down Expand Up @@ -720,7 +727,6 @@ def is_suite_running(self, user_name, suite_name, hosts=None):
"host": host,
"reason_key": self.REASON_KEY_FILE,
"reason_value": line})
break
if host_proc_dict:
sleep(0.1)

Expand Down
8 changes: 5 additions & 3 deletions t/rose-suite-clean/01-running.t
Original file line number Diff line number Diff line change
Expand Up @@ -32,15 +32,17 @@ NAME=$(basename $SUITE_RUN_DIR)
# Install suite, and prove that directories are created
rose suite-run --debug -q \
-C $TEST_SOURCE_DIR/$TEST_KEY_BASE --name=$NAME --no-gcontrol
SUITE_PROC=$(pgrep -u$USER -fl "python.*cylc-run.*\\<$NAME\\>")
ls -ld $HOME/cylc-run/$NAME 1>/dev/null
poll ! test -e $SUITE_RUN_DIR/log/job/my_task_1.2013010100.1
SUITE_PROC=$(pgrep -u$USER -fl "python.*cylc-run.*\\<$NAME\\>" \
| awk '{print "[FAIL] " $0}')
#-------------------------------------------------------------------------------
TEST_KEY=$TEST_KEY_BASE-running
run_fail "$TEST_KEY" rose suite-clean -y $NAME
file_cmp "$TEST_KEY.err" "$TEST_KEY.err" <<__ERR__
[FAIL] Suite "$NAME" may still be running.
[FAIL] Host "localhost" has process:
[FAIL] $SUITE_PROC
$SUITE_PROC
[FAIL] Try "rose suite-shutdown --name=$NAME" first?
__ERR__
if [[ ! -d $HOME/cylc-run/$NAME ]]; then
Expand All @@ -52,7 +54,7 @@ run_fail "$TEST_KEY" rose suite-clean -y -n $NAME
file_cmp "$TEST_KEY.err" "$TEST_KEY.err" <<__ERR__
[FAIL] Suite "$NAME" may still be running.
[FAIL] Host "localhost" has process:
[FAIL] $SUITE_PROC
$SUITE_PROC
[FAIL] Try "rose suite-shutdown --name=$NAME" first?
__ERR__
if [[ ! -d $HOME/cylc-run/$NAME ]]; then
Expand Down
4 changes: 2 additions & 2 deletions t/rose-suite-log/01-update-cycle/suite.rc
Original file line number Diff line number Diff line change
Expand Up @@ -5,8 +5,8 @@
timeout handler = "rose suite-hook --shutdown"
timeout = 1
[scheduling]
initial cycle time = 2013010100
final cycle time = 2013010200
initial cycle point = 2013010100
final cycle point = 2013010200
[[dependencies]]
[[[ 0, 12 ]]]
graph = """
Expand Down
4 changes: 2 additions & 2 deletions t/rose-suite-log/02-update-force/suite.rc
Original file line number Diff line number Diff line change
Expand Up @@ -5,8 +5,8 @@
timeout handler = "rose suite-hook --shutdown"
timeout = 1
[scheduling]
initial cycle time = 2013010100
final cycle time = 2013010200
initial cycle point = 2013010100
final cycle point = 2013010200
[[dependencies]]
[[[ 0, 12 ]]]
graph = """
Expand Down
8 changes: 3 additions & 5 deletions t/rose-suite-run/00-run-basic.t
Original file line number Diff line number Diff line change
Expand Up @@ -40,16 +40,14 @@ NAME=$(basename $SUITE_RUN_DIR)
run_pass "$TEST_KEY" \
rose suite-run -C $TEST_SOURCE_DIR/$TEST_KEY_BASE --name=$NAME --no-gcontrol
HOST=$(<$SUITE_RUN_DIR/log/rose-suite-run.host)
TIMEOUT=$(($(date +%s) + 60)) # wait 1 minute
while [[ ! -e $HOME/.cylc/ports/$NAME ]] && (($(date +%s) < TIMEOUT)); do
sleep 1
done
poll ! test -e $SUITE_RUN_DIR/log/job/my_task_1.2013010100.1
if [[ $HOST == 'localhost' ]]; then
SUITE_PROC=$(pgrep -u$USER -fl "python.*cylc-run.*\\<$NAME\\>")
else
CMD_PREFIX="ssh -oBatchMode=yes $HOST"
SUITE_PROC=$($CMD_PREFIX "pgrep -u\$USER -fl 'python.*cylc-run.*\\<$NAME\\>'")
fi
SUITE_PROC=$(awk '{print "[FAIL] " $0}' <<<"$SUITE_PROC")
#-------------------------------------------------------------------------------
# "rose suite-run" should not work while suite is running.
# except --reload mode.
Expand All @@ -60,7 +58,7 @@ for OPTION in -i -l '' --restart; do
file_cmp "$TEST_KEY.err" "$TEST_KEY.err" <<__ERR__
[FAIL] Suite "$NAME" may still be running.
[FAIL] Host "${HOST:-localhost}" has process:
[FAIL] $SUITE_PROC
$SUITE_PROC
[FAIL] Try "rose suite-shutdown --name=$NAME" first?
__ERR__
done
Expand Down
4 changes: 2 additions & 2 deletions t/rose-suite-run/00-run-basic/suite.rc
Original file line number Diff line number Diff line change
Expand Up @@ -5,8 +5,8 @@
timeout handler=rose suite-hook --shutdown
timeout=2
[scheduling]
initial cycle time=2013010100
final cycle time=2013010212
initial cycle point=2013010100
final cycle point=2013010212
[[dependencies]]
[[[ 0, 12 ]]]
graph=my_task_1
Expand Down
4 changes: 2 additions & 2 deletions t/rose-suite-run/02-install/suite.rc
Original file line number Diff line number Diff line change
Expand Up @@ -5,8 +5,8 @@
timeout handler = "rose suite-hook --shutdown"
timeout = 2
[scheduling]
initial cycle time = 2013010100
final cycle time = 2013010200
initial cycle point = 2013010100
final cycle point = 2013010200
[[dependencies]]
[[[ 0, 12 ]]]
graph = """
Expand Down
4 changes: 2 additions & 2 deletions t/rose-suite-run/05-log/suite.rc
Original file line number Diff line number Diff line change
Expand Up @@ -5,8 +5,8 @@
timeout handler = "rose suite-hook --shutdown"
timeout = 1
[scheduling]
initial cycle time = 2013010100
final cycle time = 2012010100 # deliberate, so no task is run
initial cycle point = 2013010100
final cycle point = 2012010100 # deliberate, so no task is run
[[dependencies]]
[[[0]]]
graph = """
Expand Down
10 changes: 7 additions & 3 deletions t/rose-suite-run/08-pgrep.t
Original file line number Diff line number Diff line change
Expand Up @@ -47,10 +47,8 @@ TIME_OUT=$(($(date +%s) + 120))
GREP="grep -q CYLC_JOB_EXIT= ~/cylc-run/$NAME/log/job/my_task_1.1.1.status"
if [[ -n $HOST ]]; then
CMD_PREFIX="ssh -oBatchMode=yes $HOST"
SUITE_PROC=$($CMD_PREFIX "pgrep -u\$USER -fl 'python.*cylc-run.*\\<$NAME\\>'")
else
CMD_PREFIX=eval
SUITE_PROC=$(pgrep -u$USER -fl "python.*cylc-run.*\\<$NAME\\>")
fi
while ! $CMD_PREFIX "$GREP" 2>/dev/null; do
if (($(date +%s) > $TIME_OUT)); then
Expand All @@ -60,13 +58,19 @@ while ! $CMD_PREFIX "$GREP" 2>/dev/null; do
done
$CMD_PREFIX "mv ~/.cylc/ports/$NAME $NAME.port"
ERR_HOST=${HOST:-localhost}
if [[ -n $HOST ]]; then
SUITE_PROC=$($CMD_PREFIX "pgrep -u\$USER -fl 'python.*cylc-run.*\\<$NAME\\>'")
else
SUITE_PROC=$(pgrep -u$USER -fl "python.*cylc-run.*\\<$NAME\\>")
fi
SUITE_PROC=$(awk '{print "[FAIL] " $0}' <<<"$SUITE_PROC")
run_fail "$TEST_KEY" \
rose suite-run -C $TEST_SOURCE_DIR/$TEST_KEY_BASE --name=$NAME \
$OPT_HOST --no-gcontrol
file_cmp "$TEST_KEY.err" "$TEST_KEY.err" <<__ERR__
[FAIL] Suite "$NAME" may still be running.
[FAIL] Host "${HOST:-localhost}" has process:
[FAIL] $SUITE_PROC
$SUITE_PROC
[FAIL] Try "rose suite-shutdown --name=$NAME" first?
__ERR__
run_pass "$TEST_KEY.NAME1" \
Expand Down
4 changes: 2 additions & 2 deletions t/rose-suite-run/10-import-1/hello/suite.rc
Original file line number Diff line number Diff line change
Expand Up @@ -5,8 +5,8 @@
timeout handler=rose suite-hook --shutdown
timeout=2
[scheduling]
initial cycle time=2013010100
final cycle time=2013010200
initial cycle point=2013010100
final cycle point=2013010200
[[dependencies]]
[[[ 0, 12 ]]]
graph = """
Expand Down
3 changes: 2 additions & 1 deletion t/rose-suite-run/14-reload-null.t
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,8 @@ poll ! test -e "$SUITE_RUN_DIR/log/job/t1.2013010100.1.status"
#-------------------------------------------------------------------------------
TEST_KEY="$TEST_KEY_BASE-0"
run_pass "$TEST_KEY" rose suite-run --run=reload -n $NAME --no-gcontrol -C src
sed '1,/export ROSE_VERSION=/d' "$TEST_KEY.out" >"$TEST_KEY.out.tail"
sed '1,/export ROSE_VERSION=/d; /^\[INFO\] delete: .*\/.cylc\//d' \
"$TEST_KEY.out" >"$TEST_KEY.out.tail"
file_cmp "$TEST_KEY.out" "$TEST_KEY.out.tail" <<__OUT__
[INFO] $NAME: reload complete. "suite.rc" unchanged
__OUT__
Expand Down
4 changes: 2 additions & 2 deletions t/rose-suite-run/14-reload-null/suite.rc
Original file line number Diff line number Diff line change
Expand Up @@ -5,8 +5,8 @@ UTC mode=True
timeout handler=rose suite-hook --shutdown
timeout=2
[scheduling]
initial cycle time=2013010100
final cycle time=2013010112
initial cycle point=2013010100
final cycle point=2013010112
[[dependencies]]
[[[0,12]]]
graph="""t1[T-12]=>t1"""
Expand Down
3 changes: 2 additions & 1 deletion t/rose-suite-run/15-reload-rc.t
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,8 @@ cat >src/rose-suite.conf <<'__CONF__'
ROSE_TASK_RUN_ARGS="-O earth"
__CONF__
run_pass "$TEST_KEY" rose suite-run --run=reload -n $NAME --no-gcontrol -C src
sed '1,/export ROSE_VERSION=/d' "$TEST_KEY.out" >"$TEST_KEY.out.tail"
sed '1,/export ROSE_VERSION=/d; /^\[INFO\] delete: .*\/.cylc\//d' \
"$TEST_KEY.out" >"$TEST_KEY.out.tail"
file_cmp "$TEST_KEY.out" "$TEST_KEY.out.tail" <<__OUT__
[INFO] delete: suite.rc
[INFO] install: suite.rc
Expand Down
Loading