nuxeo-distribution-dm/ftest/funkload/README.txt
author Solen Guitter <sguitter@nuxeo.com>
Mon, 07 Mar 2011 11:43:45 +0100
branch5.4
changeset 2008 57fbd264dd17
parent 895 86911232b2ee
permissions -rw-r--r--
NXP-6465: fixed labels
     1 ======================================
     2 Funkload Nuxeo DM jboss test scripts
     3 ======================================
     4 
     5 Abstract
     6 ============
     7 
     8 This module contains python funkload scripts. These scripts can be used
     9 to do functional testing or benching of the default Nuxeo DM web interface
    10 including rest api implemented in nuxeo-platform-ui-web.
    11 
    12 
    13 Script example
    14 ==============
    15 
    16 Testing rest api
    17 -----------------
    18 
    19 Using the RestAPI helper class::
    20 
    21  class MySuite(FunkLoadTestCase):
    22      def testRest(self):
    23          r = RestAPI(self)
    24          r.login('Administrator', 'Administrator')
    25          root_uid = r.getRootWorkspaceUid()         # get the root uid
    26          ws_uid = r.createDocument(root_uid,
    27                                    'Workspace', 'a rest workspace',
    28                                    'A description')
    29          r.logout()
    30 
    31 Visit test_rest.py for more examples.
    32 
    33 Testing the web interface
    34 ---------------------------
    35 
    36 These test use a kind of `PageObject
    37 <http://code.google.com/p/webdriver/wiki/PageObjects>`_ and `fluent
    38 interface <http://www.martinfowler.com/bliki/FluentInterface.html>`_
    39 patterns.
    40 
    41 Writing a new scenario can be done like this::
    42 
    43  class MySuite(NuxeoTestCase):
    44     def testMyScenario(self):
    45         (LoginPage(self)
    46          .login('Administrator', 'Administrator')
    47          .getRootWorkspaces()
    48          .createWorkspace('My workspace', 'Test ws')
    49          .rights().grant('ReadWrite', 'members')
    50          .view()
    51          .createFolder('My folder', 'Test folder')
    52          .createFile('My file', 'Test file', 'foo.pdf')
    53          .getRootWorkspaces().deleteItem("My workspace")
    54          .logout())
    55 
    56 
    57 Visit test_pages.py for more examples.
    58 
    59 Layout
    60 =======
    61 
    62 * nuxeo/rest.py
    63 
    64   RestAPI class helper.
    65 
    66 * nuxeo/pages.py
    67 
    68   The Pages class which are simple abstraction of the web app, the main
    69   difference with web driver PageObjects is that they contains assertion to
    70   ease the fluent interface.
    71 
    72 * nuxeo/testcase.py
    73 
    74   Simple nuxeo test case that use an xmlrpc server to get credential to
    75   access the nuxeo ep.
    76 
    77 * test_rest.py, Rest.conf
    78 
    79   Test suite to check rest api (nuxeo/rest.py):
    80   Create a workspace, a folder and few documents, ExportTree on few documents.
    81 
    82 * test_pages.py, Page.conf
    83 
    84   A test suite to check the pages api (nuxeo/pages.py). Note that the test
    85   suite will create a new member account named flnxtest.
    86 
    87 * test_nuxeo.py, Nuxeo.conf
    88 
    89   A test suite used to test/bench a Nuxeo DM. This test will create many
    90   members accounts (the ones present in the password.txt files) and will add
    91   a new workspace and folder with few file documents.
    92 
    93 * credential.conf, passwd.txt, groups.txt
    94 
    95   Credential xmlrpc configuration with unix like passwd/groups files used by
    96   nuxeo testcase to get credentials.
    97 
    98 * monitor.conf
    99 
   100   Configuration of the local monitor server used by the bench runner.
   101 
   102 Running tests
   103 ===============
   104 
   105 
   106 Requirement
   107 -------------
   108 
   109 * Install latest FunkLoad snapshots, on Lenny or Intrepid Ibex this can be
   110   done like this::
   111 
   112   sudo aptitude install python-dev python-xml python-setuptools \
   113        python-webunit=1.3.8-1 python-docutils gnuplot
   114   sudo aptitude install tcpwatch-httpproxy --without-recommends
   115   sudo easy_install -f http://funkload.nuxeo.org/snapshots/ -U funkload
   116 
   117 
   118 For other distro visit http://funkload.nuxeo.org/INSTALL
   119 
   120 
   121 Running test
   122 -------------
   123 
   124 Using a makefile
   125 
   126 * make
   127   this is equivalent to "make start rest page test-nuxeo stop"
   128 
   129 * make start
   130   Start local monitoring
   131   Start the xmlrpc credential server to serve user/password to test runner.
   132   Check if nuxeo ep login page is accessible.
   133 
   134 * make stop
   135   Stop monitoring, credential server.
   136 
   137 * make rest
   138   Run  the test_rest suite.
   139 
   140 * make page
   141   Run the test_pages suite.
   142 
   143 * make test-nuxeo
   144   Run the test_nuxeo suite.
   145 
   146 To run test on remote site use the URL option:
   147 
   148   make rest URL=http://host.to.nuxeo:8080/nuxeo
   149 
   150 To pass extra options to the funkload test runner using EXT:
   151 
   152   make page EXT="-dV"
   153 
   154   This will run the test in debug mode and try to ouput response page in a
   155   running Firefox (See fl-run-test -h for more information).
   156 
   157 Note that log files are located in the target/ftest/funkload of the
   158 nuxeo-distribution-dm.
   159 
   160 Benching
   161 ---------
   162 
   163 * make bench
   164   Full bench: init users and test layout, bench writer, bench reader.
   165 
   166   This will produce an html report located in
   167   ../../target/ftest/funkload/report
   168 
   169 * make bench-reader
   170   Bench the reader part only.
   171 
   172 
   173 
   174 Nuxeo DM running on JAVA 5
   175 --------------------------
   176 
   177 Starting with Nuxeo DM 5.3.0 the default JVM is Sun Java 6.
   178 
   179 Scripts must be adapted to use JAVA5.
   180 
   181 Memory and PostgreSQL analysis per page
   182 ------------------------------------------
   183 
   184 By install JMXSH and having access to the PostgreSQL log file, the Nuxeo
   185 test case can collect and extract meaningful metrics for each page.
   186 
   187 Requirement:
   188 
   189 * Install JMXSH from http://code.google.com/p/jmxsh/ and make
   190 the jar available at /usr/local/jmxsh/jmxsh-R5.jar or set a JMXSH
   191 environment variable.
   192 
   193 * Install pgfouine, gawk
   194 
   195 * Enable JMX access by setting the java option in the nuxeo.conf
   196 
   197    JAVA_OPTS=$JAVA_OPTS -Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote.port=1089 -Dcom.sun.management.jmxremote.ssl=false -Dcom.sun.management.jmxremote.authenticate=false
   198 
   199 * Edit the FunkLoad Nuxeo.conf file and set the following path:
   200 
   201 pglog=/path/to/postgresql/log/postgresql-8.4-main.log
   202 log_dir=../../target/ftest/funkload/log
   203 monitorctl_file=/path/to/nuxeo-server/bin/monitorctl.sh
   204 
   205 * To extract all the SQL queries you need to use the following
   206   PostgreSQL setup:
   207 
   208      lc_messages = 'en_US.UTF-8'
   209      log_line_prefix = '%t [%p]: [%l-1] '
   210      log_min_duration_statement = 0
   211 
   212 Usage:
   213 
   214 Run your test, the FunkLoad Nuxeo test case will perform a Full GC and
   215 a heap history before each request and will save all SQL queries and
   216 heap history after.
   217 
   218 You can generate the pgfouine reports per page like this:
   219  
   220 for f in `ls pg-test*.log`; do pgfouine -file $f -logtype stderr -top 30 > ${f%log}html; done
   221 
   222 You can view the amount of memory needed by page like this:     
   223 
   224 for f in `ls hh-test*-before.txt`; do e=${f%-before.txt}-end.txt; after=`grep Total $e | awk '{ print $3 }'`; before=`grep Total $f | awk '{ print $3 }'`; diff=`expr $after - $before`; hrdiff=`echo $diff | awk '{sum=$1;hum[1024**3]="Gb";hum[1024**2]="Mb";hum[1024]="Kb"; for (x=1024**3; x>=1024; x/=1024){ if (sum>=x) { printf "%.2f %s\n",sum/x,hum[x];break }}}'`; echo "${f%-before.txt} $before $after: $hrdiff"; done
   225 
   226 Note that you need to use a huge heap to prevent any minor or major GC.