Skip to content

Test environment for content

Jump to bottom
Frédéric Wang edited this page Jan 9, 2013 · 1 revision

Test Environment for content

Overview

A key MathJax goal is further the adoption of MathML in HTML. Content producers are still hesitant to output MathML in HTML and prefer images despite using an internal XML/MathML workflow.

Creating a test environment for authors to test their content will allow users to reliably switch to MathML output.

Additionally, content testing could allow content producers to find optimal configurations.

  • Hooks for different errors are already in place
    • command error
    • fonts missing
    • character missing
  • configuration checker
  • Possible solutions:
    • create a configuration that focuses on these signals
    • create data collector for these
    • Additionally: hook up multiple pages via iframes -> generate report for larger projects
  • should we offer this as a webservice? What could be considered?
    • Create frontend for
      • individual equations (copy & paste)
      • HTML+TeX, HTML+MathML
      • TeX
    • Like a validator?
    • should we offer a js-tool that would allow to open gateway to test content on other people's server?
  • How about feedback to mathjax so that we get to know more limitations, can add new features?
  • This is related to platform testing (desktop, mobile, apps)

Web Service

Practical considerations

Basic tests can be done on the client's browser and so could be directly implemented as Javascript on Web pages and made available from the CDN. In general, we need a Web server with programming capabilities in order to perform more advanced verifications of Web contents. The most obvious option is to rely on the Amazon EC2 machines already configured. In particular, we will have to consider a trade-off between EC2 cost and services proposed:

  • The Web Service must be permanently available. This means to reserve at least a micro Linux machine for one year (Heavy Utilization Reserved Instances is 62$ ; US East: $0.005 * 365 * 24 = $43.8 do not exceed the upfront). However, additional charges apply if the service is used a lot. That machine could also be used to permanently serve http://devel.mathjax.org/ with the MathJax test suite page and MathJax development branches.

  • Platforms: if we need to use Selenium to load the page in a browser, we can only do it in Opera, Firefox and Chrome on Linux. To test Windows and in particular Internet Explorer, we will need another machine. Safari is not supported yet. Support for mobile platforms (iOS and Android) is not implemented and may not be possible on Amazon EC2.

  • The testing may be slow, especially if we use a micro machine only. To consider:

    • Restricted access? Perhaps only propose basic features and more advanced features according to sponsorship level.
    • Limit the input (size of source code, number of URLs...)
    • Run some tests on the client's browser. This limits the testing (client's configuration & platform, no UI tests), is not convenient for the clients (use its computer resources), may not be allowed by browser's security restriction (e.g. cross-domain), can not handle things like browser crashes/hangs etc
    • Like in the test framework, use powerful test machines if we need to load pages in browsers (note: automatic launch/termination of machines is not implemented yet)
    • Semi-automated testing: when [Math Processing Error], crashes etc are found, report them to the MathJax developers who will try to find out the issue.
    • For tests that require a long processing, use a waiting queue to handle the user's requests and provide the results later by mails when they are available.
    • Rely on efforts to extract MathJax components (parsers, converters) and make them independent of the browsers, to avoid having to use a browser.

Potential Features

Note: I stretched the list a bit outside the scope of "testing".

  • Basic features that can be made available on the CDN:

    • Basic examples like http://cdn.mathjax.org/mathjax/latest/test/examples.html Can be done directly on a Web page.

    • Basic performance test checking that like Sean Hogan's tests. Can be done directly on a Web page.

    • Given a AsciiMathML/LaTeX/MathML equation (pasted in a Web form), parse it using MathJax and render it, or report parsing or Math Processing errors. Can be done directly on a Web page.

    • Given an AsciiMathML/LaTeX equation (pasted into a Web form), convert it into MathML using MathJax. Can be done directly on a Web page.

    • On-line math editor. See Wysiwyg math editors.

  • Basic features that could be done on the CDN or otherwise using a Linux machine:

    • Given a MathML equation (pasted into a form), validate it against the MathML RelaxNG schema. Can maybe be done directly on a Web page (see e.g http://debeissat.nicolas.free.fr/relaxng.php) or using jing on the EC2 machine.

    • Given a content MathML equation (pasted into a form), convert it into presentation MathML. Use David's XSLT stylesheet. Can maybe be done directly on a Web page (see e.g. http://debeissat.nicolas.free.fr/xsl_tests.php), using PHP, or using xsltproc on the EC2 machine.

    • Given a MathML equation (pasted into a Web form), convert it to LaTeX or ASCII-art. See Davide Carlisle's pmml2tex and pmathmlascii. Use XSLT convertor as above. Note that these are XSLT 2.0 stylesheet which may not be implemented in browsers.

    • Given a document (loaded from a form) convert it into Web pages (which will use MathJax) and let the user download a zip archive of the output. There are several LaTeX to MathML converters, LaTeXML has a pretty good LaTeX support and an "include MathJax" option They have tex2xml service, latex2mathml or ltxMojo. But their services seem to be unavailable sometimes and I don't think the MathJax option is enabled by default. See also Document Preparation. These programs could be executed on the EC2 machine.

      • Test case reduction tool (we don't have such a tool yet). Given a MathML fragment or a Web page, produce a reduce test case that exhibits the error. Search in the issue tracker the MathML element/attribute names, LaTeX commands etc from the reduce testcases, in order to point to known bugs. The basic idea would be the ddmin algorithm which requires to load smaller and smaller subsets of the page in the browser. This is likely to take some time, so can not be done on a micro EC2 machine...

  • Page validator. The user provides a page (or a set of pages?) from its URL or a local file and we perform some verifications on it.

    • The verification can either be static (we download the source of the page and use scripts to verify the XML tree) or dynamic (we open the page in a browser so that MathJax is really executed). For dynamic tests, we can either use Selenium and let the user choose among a list of browsers/operating systems or use an iframe where the pages are loaded one after the other. However, see the comment above about the limitations of the second approach.

    • Validate the MathML equations inside the page against the RelaxNG schema (static)

    • Load the page in the browser, get the sources of the MathML fragments generated by MathJax and validate against the RelaxNG schema (dynamic)

    • Check whether AsciiMath/LaTeX/MathML is present on a page and compare against the configuration options actually used. Same for extensions. Give a warning if the page seems to use too much or not enough. (static)

    • Load the page in a browser, check that MathJax is loaded and completes. Report time to load page and possible MathJax errors (dynamic)

      • If there are LaTeX command errors: suggest missing configuration options

      • If there are [Math Processing Error], crashes etc Apply the Test case reduction tool.

      • Screen capture of the page.

    • Same, but once MathJax completes, perform random UI actions using Selenium.

MathJax Wiki

Clone this wiki locally