Skip to content

Commit

Permalink
Revise tools scripts to be python3 compatible on win32
Browse files Browse the repository at this point in the history
We introduce setup_stdio function to setup stdout/stderr properly.
For python <-> python pipe, we always use 'utf8'/'ignore' encoding for not lost
characters.
For tty <-> python, we using native encoding with xmlcharrefreplace to encode, to
preserve maximal information.
For python <-> native program, we use naive encoding with 'ignore' to not cause error

update_exclude_list with binary mode so that on win32 would not generate \r\n

run-test-suite.py: Handling skiplist properly on win32

Fixes jerryscript-project#4854

Fixes test262-harness.py complain cannot use a string pattern on a bytes-like object with running test262 with python3

For reading/writing to file, we use 'utf8' /'ignore' encoding for not lost characters.
For decoding from process stdout, using native encoding with decoding error ignored for not lost data.
Execute commands also ignore errors
Fixes jerryscript-project#4853

Fixes running test262-esnext failed with installed python3.9 on win32 with space in path
Fixes jerryscript-project#4852

```
support both / \ in --test262-test-list arg

On win32.
python tools/run-tests.py  --test262-es2015=update --test262-test-list=built-ins/decodeURI/
python tools/run-tests.py  --test262-es2015=update --test262-test-list=built-ins\decodeURI\
should be both valid,
currently only --test262-test-list=built-ins\decodeURI\ are valid.
```

```
Support snapshot-tests-skiplist.txt on win32 by use os.path.normpath
```

Guard run-tests.py with timer.

All run-tests.py are finished in 30 minutes in normal situation.
May increase the timeout future.

wait JERRY_CHECK_TIMEOUT

```

Move Windows CI to github actions

Convert run-debugger-test.sh to run-debugger-test.py

After this change, run-debugger-test.py could running on Win32 and OSX

Define TERM colors for win32 properly

```

```

flush stderr.write stdout.write

On CI, the stderr are redirect to stdout, and if we don't flush stderr and stdout,
The output from stderr/stdout would out of sync.

```

JerryScript-DCO-1.0-Signed-off-by: Yonggang Luo luoyonggang@gmail.com
  • Loading branch information
lygstate committed Nov 14, 2024
1 parent f15e920 commit 0c3080f
Show file tree
Hide file tree
Showing 12 changed files with 208 additions and 137 deletions.
32 changes: 32 additions & 0 deletions .github/workflows/gh-actions.yml
Original file line number Diff line number Diff line change
Expand Up @@ -56,6 +56,36 @@ jobs:
- run: $RUNNER -q --jerry-tests --buildoptions=--compile-flag=-m32,--cpointer-32bit=on
- run: $RUNNER -q --jerry-tests --buildoptions=--compile-flag=-m32,--cpointer-32bit=on --build-debug

Win_x86-64_Conformance_Tests_ESNext:
runs-on: windows-latest
steps:
- uses: actions/checkout@v2
- run: python $env:RUNNER --test262 update

Win_x86-64_Conformance_Tests_ESNext_Debug:
runs-on: windows-latest
steps:
- uses: actions/checkout@v2
- run: python $env:RUNNER --test262 update --build-debug

Win_x86-64_Tests:
runs-on: windows-latest
steps:
- uses: actions/checkout@v2
- run: python $env:RUNNER -q --jerry-tests
- run: python $env:RUNNER -q --unittests
- run: python $env:RUNNER -q --buildoption-test
- run: python $env:RUNNER -q --jerry-debugger

Win_x86-64_Tests_Debug:
runs-on: windows-latest
steps:
- uses: actions/checkout@v2
- run: python $env:RUNNER -q --jerry-tests --build-debug
- run: python $env:RUNNER -q --unittests --build-debug
- run: python $env:RUNNER -q --buildoption-test --build-debug
- run: python $env:RUNNER -q --jerry-debugger --build-debug

OSX_x86-64_Build_Correctness_Unit_Tests:
runs-on: macos-13
steps:
Expand All @@ -65,6 +95,7 @@ jobs:
python-version: '>=3.6'
- run: $RUNNER -q --jerry-tests
- run: $RUNNER -q --unittests
- run: $RUNNER -q --jerry-debugger

OSX_x86-64_Build_Correctness_Unit_Tests_Debug:
runs-on: macos-13
Expand All @@ -75,6 +106,7 @@ jobs:
python-version: '>=3.6'
- run: $RUNNER -q --jerry-tests --build-debug
- run: $RUNNER -q --unittests --build-debug
- run: $RUNNER -q --jerry-debugger --build-debug

Linux_x86-64_Build_Option_Tests:
runs-on: ubuntu-latest
Expand Down
27 changes: 0 additions & 27 deletions appveyor.yml

This file was deleted.

2 changes: 2 additions & 0 deletions jerry-debugger/jerry_client.py
Original file line number Diff line number Diff line change
Expand Up @@ -326,6 +326,8 @@ def main():
break
if res_type == result.PROMPT:
prompt.cmdloop()
sys.stdout.flush()
sys.stderr.flush()
elif res_type == result.TEXT:
write(result.get_text())
continue
Expand Down
43 changes: 24 additions & 19 deletions tools/run-tests.py
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,9 @@

OUTPUT_DIR = os.path.join(settings.PROJECT_DIR, 'build', 'tests')

# All run_check proc must finished in 30 minutes, may increase in future
JERRY_CHECK_TIMEOUT = 30 * 60

Options = collections.namedtuple('Options', ['name', 'build_args', 'test_args', 'skip'])
Options.__new__.__defaults__ = ([], [], False)

Expand Down Expand Up @@ -197,25 +200,22 @@ def get_arguments():

BINARY_CACHE = {}

TERM_NORMAL = '\033[0m'
TERM_YELLOW = '\033[1;33m'
TERM_BLUE = '\033[1;34m'
TERM_RED = '\033[1;31m'

def report_command(cmd_type, cmd, env=None):
sys.stderr.write(f'{TERM_BLUE}{cmd_type}{TERM_NORMAL}\n')
sys.stderr.write(f'{util.TERM_BLUE}{cmd_type}{util.TERM_NORMAL}\n')
if env is not None:
sys.stderr.write(''.join(f'{TERM_BLUE}{var}={val!r} \\{TERM_NORMAL}\n'
sys.stderr.write(''.join(f'{util.TERM_BLUE}{var}={val!r} \\{util.TERM_NORMAL}\n'
for var, val in sorted(env.items())))
sys.stderr.write(f"{TERM_BLUE}" +
f" \\{TERM_NORMAL}\n\t{TERM_BLUE}".join(cmd) +
f"{TERM_NORMAL}\n")
sys.stderr.write(f"{util.TERM_BLUE}" +
f" \\{util.TERM_NORMAL}\n\t{util.TERM_BLUE}".join(cmd) +
f"{util.TERM_NORMAL}\n")
sys.stderr.flush()

def report_skip(job):
sys.stderr.write(f'{TERM_YELLOW}Skipping: {job.name}')
sys.stderr.write(f'{util.TERM_YELLOW}Skipping: {job.name}')
if job.skip:
sys.stderr.write(f' ({job.skip})')
sys.stderr.write(f'{TERM_NORMAL}\n')
sys.stderr.write(f'{util.TERM_NORMAL}\n')
sys.stderr.flush()

def create_binary(job, options):
build_args = job.build_args[:]
Expand Down Expand Up @@ -246,13 +246,15 @@ def create_binary(job, options):
if binary_key in BINARY_CACHE:
ret, build_dir_path = BINARY_CACHE[binary_key]
sys.stderr.write(f'(skipping: already built at {build_dir_path} with returncode {ret})\n')
sys.stderr.flush()
return ret, build_dir_path

try:
subprocess.check_output(build_cmd)
ret = 0
except subprocess.CalledProcessError as err:
print(err.output.decode("utf8"))
# For python <-> native program, we use default encoding with error='ignore' to not lost data
print(err.output.decode(errors="ignore"))
ret = err.returncode

BINARY_CACHE[binary_key] = (ret, build_dir_path)
Expand Down Expand Up @@ -283,6 +285,7 @@ def iterate_test_runner_jobs(jobs, options):

if build_dir_path in tested_paths:
sys.stderr.write(f'(skipping: already tested with {build_dir_path})\n')
sys.stderr.flush()
continue
tested_paths.add(build_dir_path)

Expand All @@ -291,6 +294,7 @@ def iterate_test_runner_jobs(jobs, options):

if bin_hash in tested_hashes:
sys.stderr.write(f'(skipping: already tested with equivalent {tested_hashes[bin_hash]})\n')
sys.stderr.flush()
continue
tested_hashes[bin_hash] = build_dir_path

Expand All @@ -308,23 +312,23 @@ def run_check(runnable, env=None):
env = full_env

with subprocess.Popen(runnable, env=env) as proc:
proc.wait()
proc.wait(timeout=JERRY_CHECK_TIMEOUT)
return proc.returncode

def run_jerry_debugger_tests(options):
ret_build = ret_test = 0
for job in DEBUGGER_TEST_OPTIONS:
ret_build, build_dir_path = create_binary(job, options)
if ret_build:
print(f"\n{TERM_RED}Build failed{TERM_NORMAL}\n")
print(f"\n{util.TERM_RED}Build failed{util.TERM_NORMAL}\n")
break

for channel in ["websocket", "rawpacket"]:
for test_file in os.listdir(settings.DEBUGGER_TESTS_DIR):
if test_file.endswith(".cmd"):
test_case, _ = os.path.splitext(test_file)
test_case_path = os.path.join(settings.DEBUGGER_TESTS_DIR, test_case)
test_cmd = [
test_cmd = util.get_python_cmd_prefix() + [
settings.DEBUGGER_TEST_RUNNER_SCRIPT,
get_binary_path(build_dir_path),
channel,
Expand Down Expand Up @@ -379,7 +383,7 @@ def run_test262_test_suite(options):
for job in jobs:
ret_build, build_dir_path = create_binary(job, options)
if ret_build:
print(f"\n{TERM_RED}Build failed{TERM_NORMAL}\n")
print(f"\n{util.TERM_RED}Build failed{util.TERM_NORMAL}\n")
break

test_cmd = util.get_python_cmd_prefix() + [
Expand Down Expand Up @@ -409,7 +413,7 @@ def run_unittests(options):
continue
ret_build, build_dir_path = create_binary(job, options)
if ret_build:
print(f"\n{TERM_RED}Build failed{TERM_NORMAL}\n")
print(f"\n{util.TERM_RED}Build failed{util.TERM_NORMAL}\n")
break

if sys.platform == 'win32':
Expand Down Expand Up @@ -438,14 +442,15 @@ def run_buildoption_test(options):

ret, _ = create_binary(job, options)
if ret:
print(f"\n{TERM_RED}Build failed{TERM_NORMAL}\n")
print(f"\n{util.TERM_RED}Build failed{util.TERM_NORMAL}\n")
break

return ret

Check = collections.namedtuple('Check', ['enabled', 'runner', 'arg'])

def main(options):
util.setup_stdio()
checks = [
Check(options.check_signed_off, run_check, [settings.SIGNED_OFF_SCRIPT]
+ {'tolerant': ['--tolerant'], 'gh-actions': ['--gh-actions']}.get(options.check_signed_off, [])),
Expand Down
99 changes: 99 additions & 0 deletions tools/runners/run-debugger-test.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,99 @@
#!/usr/bin/env python

# Copyright JS Foundation and other contributors, http://js.foundation
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

import os
import subprocess
import sys
import time

import util
TempFile = __import__("test262-harness").TempFile # pylint: disable=invalid-name

class DebuggerArgs:
def __init__(self):
self.jerry = sys.argv[1]
self.channel = sys.argv[2]
self.debugger_client = sys.argv[3]
self.test_case = sys.argv[4]


def check_output(command_args, stdin=None, encoding=None):
try:
out = subprocess.check_output(command_args, stdin=stdin, shell=False, stderr=subprocess.STDOUT)
except subprocess.CalledProcessError as check_error:
out = check_error.output
return out.decode(encoding or 'utf-8', 'ignore')


def execute_debug_client(out_tmp, cmd_file_name, debug_client_args):
print(f'input debug cmd: {cmd_file_name}')
with open(cmd_file_name, 'rb') as cmd_file:
out = check_output(debug_client_args, cmd_file)
out_tmp.write(out)


def main(args):
util.setup_stdio()
jerry_debug_server_cmd = [args.jerry]
client_args = []
if 'client_source' in args.test_case:
jerry_debug_server_cmd += ['--start-debug-server', '--debug-channel',
args.channel, '--debugger-wait-source']
client_args += ['--client-source']
if 'client_source_multiple' in args.test_case:
client_args += [args.test_case + '_2.js', args.test_case + '_1.js']
else:
client_args += [args.test_case + '.js']
else:
jerry_debug_server_cmd += [args.test_case + '.js', '--start-debug-server', '--debug-channel', args.channel]
print(f'run debug server: {jerry_debug_server_cmd}')
with subprocess.Popen(jerry_debug_server_cmd, stdin=subprocess.PIPE,
stdout=subprocess.PIPE, stderr=subprocess.STDOUT) as proc:
time.sleep(1)

out_tmp = TempFile(prefix=os.path.basename(args.test_case), suffix='out')
git_failed = False
try:
debug_client_args = util.get_python_cmd_prefix()
debug_client_args += [args.debugger_client, '--channel', args.channel, '--non-interactive']
debug_client_args += client_args
print(f"run debug client: {' '.join(debug_client_args)}")
execute_debug_client(out_tmp, args.test_case + '.cmd', debug_client_args)
if 'restart' in args.test_case:
continue_case = args.test_case.replace('restart', 'continue')
execute_debug_client(out_tmp, continue_case + '.cmd', debug_client_args)
out_tmp.close()
git_diff_cmd = ['git', '--no-pager', 'diff', '--ignore-space-at-eol',
'--no-index', args.test_case + '.expected', out_tmp.name]
git_out = check_output(git_diff_cmd)
if '@@' in git_out:
git_failed = True
finally:
proc.wait()
print(f"jerry out:\n{proc.stdout.read().decode('utf-8')}\nEOF")
print(f"git diff cmd: {' '.join(git_diff_cmd)}")
if git_failed:
print(f'git diff result:\n{git_out}\nEOF')
print(f'{util.TERM_RED}FAIL: {args.test_case}{util.TERM_NORMAL}')
sys.exit(1)
else:
out_tmp.dispose()
print(f'{util.TERM_GREEN}PASS: {args.test_case}{util.TERM_NORMAL}')
sys.exit(0)


if __name__ == "__main__":
sys.exit(main(DebuggerArgs()))
66 changes: 0 additions & 66 deletions tools/runners/run-debugger-test.sh

This file was deleted.

Loading

0 comments on commit 0c3080f

Please sign in to comment.