RushCheck 0.8

A lightweight random testing tool for Ruby




Chapter 1
Getting started

There are three ways to install RushCheck.

1.1  Installing via RubyGems

You can install RushCheck easily with rubygems.

sudo gem install rushcheck


1.2  Installing from source code

Instead, if you don’t have rubygems, or like to install from source codes, then you can follow the steps.

  1. Get the tar-ball of current release from download page

  2. Expand it

  3. finally, type as follows

    sudo ruby setup.rb

See also the following:

setup.rb help

1.3  Installing from Darcs repository

Our darcs repository is provided for developer RushCheck itself, and skip this section if you want to only use RushCheck! If you have an interest to our development version (not yet shipped), then try to access our darcs repository. darcs is a version control system and can be easily installed from the darcs page. After installing darcs, you can get the current development of RushCheck by following:

darcs get --partial http://rushcheck.rubyforge.org/repos/rushcheck

After darcs getting, you can get the newest files whenever you want to type

darcs pull -a

Chapter 2

At first, we have to require the RushCheck library. If you have installed RushCheck using RubyGems, then you should add the following two lines to your test codes.

require 'rubygems'
require_gem 'rushcheck'

Otherwise, if you have installed RushCheck by setup.rb, then add the simple one line.

require 'rushcheck'

The following maybe useful if you don’t matter whether you use rubygems or not.

  require 'rubygems'
  require_gem 'rushcheck'
rescue LoadError
  require 'rushcheck'

Also, don’t forget to require your library file if the class you want to test is included in it.

require 'your_library'

2.1  Writing test cases

OK, then we can now start to write test codes. RushCheck requires you to write assertion based test cases. An assertion of function (or method) consists of triple where inputs, guard conditions and a testing property block. Here is a templete of assertion:

RushCheck::Assertion.new(Class1, Class2, ...) do |var1, var2, ...|
  # testing property block
  # this block should return boolean value (true, false or nil)
  # in Ruby

For example, suppose that we want to test the sort method of the Array class. The following assertion is a simple test case:

ast_zero =
  RushCheck::Assertion.new() do
    [].sort == []

whether if we sort empty array the result is also empty. This assertion does not have any inputs and guards but only have the property block.

Let’s see another example:

ast_one =
  RushCheck::Assertion.new(Integer) do |x|
    [x].sort == [x]

This assertion defines that we claim for any integer x, an array [x] is not changed after sorting. We can test the property 100 times:

irb> ast_one.verbose_check
... (snip) ...
OK, passed 100 tests.
irb> ast_one.check
OK, passed 100 tests.

RushCheck supports random testing such as above. We will see later how to change the number of test, how to change the distribution, etc. Here we learned two testing methods, verbose_check and check. ‘verbose_check’ displays every instances of test, on the other hand ‘check’ does not display inputs but only the result.

Next example shows how RushCheck displays the counter example. If an assertion is failed, then there is a counter example of the assertion.

ast_two =
  RushCheck::Assertion.new(Integer, Integer) do |x, y|
    [x, y].sort == [x, y]

The above test is failed sometimes and we can find the result after checking.

irb> ast_two.check
Falsifiable, after 3 tests:
[2, -1]

Here, the counter example [2, -1] of the assertion is appeared. In fact, [2, -1].sort equals [-1, 2] and does not equal [2, -1].

Sometimes, we need several pre-conditions for tests. For example, if we have a pre-condition x <= y in the previous assertion, then the assertion should be succeeded. We can write pre-conditions as guards in the property block:

ast_two_sorted =
  RushCheck::Assertion.new(Integer, Integer) do |x, y|
    RushCheck::guard { x <= y }
    [x, y].sort == [x, y]

irb> ast_two_sorted.check
OK, passed 100 tests.

Note that it is always assumed that the number of arguments of Assertion.new must be equal to the number of variables of the block. (Until ver 0.3, experimentally the number of arguments can be differed, however from ver 0.4, they must be equal.)

The arguments of Assertion.new corresponds to the variables of block one to one. We can have any number of guards in the property block. If the guard property does not hold in random testing, then the test is abort and try to take another random instances. We can imagine the following test sequences in ast_two_sorted.

  1. x, y = 1, 2

    -> guard g is passed -> check the test block and succeed

  2. x, y = 3, 1

    -> guard g is failed and abort (not counted)

  3. x, y = 2, 3

    -> …

  4. … (whether passed 100 tests or not)

2.2  Watching the statistics

In the previous section, we saw two methods check and verbose_check. Sometimes, we want to watch the statistics of random testing. However check shows less information, and verbose_check is too noisy. RushCheck has several options to watch the statistics.

2.2.1  trivial

You may not want to check the trivial case in random test. As we have seen, we can ignore the trivial case using the guard condition. However, on the other hand, we can watch how many trivial cases are appeared in random test.

ast_sort_triv =
  RushCheck::Assertion.new(Integer, Integer) do |x, y|
    RushCheck::guard { x <= y }
    ([x, y].sort == [x, y]).trivial{ x == y }

irb> ast_sort_triv.check
OK, passed 100 tests(14%, trivial).

Here, we have 14% (i.e. 14 times) trivial (x == y) cases in the test.

2.2.2  classify

In addition, we can give another names to watching statistics.

ast_sort_classify =
  RushCheck::Assertion.new(Integer, Integer) do |x, y|
    RushCheck::guard { x <= y }
    test = ([x, y].sort == [x, y])
    test.classify('same'){ x == y }.
      classify('bit diff'){ (x - y).abs == 1 }

irb> ast_sort_classify.check
OK, passed 100 tests.
18%, same.
11%, bit diff.

2.3  Integration with unit testing

The library ‘test/unit’ of Ruby is useful for unit testing. Here is a trick to use ‘test/unit’.

def forall(*cs, &f)
  assert(RushCheck::Claim.new(*cs, &f).check)

The class Claim is a subclass of Assertion. The meaning is almost similar to Assertion, however Claim does not check the result value of the given block ‘f’. Because assertions in ‘test/unit’ such as ‘assert_equal’ does not return any result, but return nil, we don’t need to check the result values of a sequence of assertions. Nevertheless we can check the test case because the assertions in ‘test/unit’ raises an exception if the test case is failed.


def test_not_empty_after_push
  array = Array.new
  forall(Integer) do |item|
    array.push item
    assert(! array.empty?)

2.4  Integration with RSpec

RSpec is another testing framework which helps Behaviour Driven Development (BDD). To combine RSpec and RushCheck, the following trick maybe useful and easy to read:

def forall(*cs, &f)
  RushCheck::Claim.new(*cs, &f).check.should_equal true

Then, for example, we can write a specification of Array#push as follows:

context "An empty array" do

  specify "should not be empty after 'push'" do
    forall(Integer) do |item|
      array = Array.new
      array.push item


See also ‘examples/rspec’ in the distribution of RushCheck for details.

2.5  Additional classes for assertions

In previous sections, we have seen how to check assertions for any integers. In similar way, we can define the assertions for any float numbers or any string.

RushCheck::Assertion.new(Float, String, ...) do |ratio, name,...|
  # test case

RushCheck has default random generators for the following basic classes:

  • Integer
  • Float
  • String

If you want to change the distribution of randomness, then you have to write a code for generator. There are some examples for writing generators. In the next section, I will introduce SpecialString whose distribution differs to the default implementation of String.

Even Array, Hash and Proc are also primitive classes, however the randomness of them should be changed in testing codes. Therefore, RushCheck provides an abstract generators for them. Programmer need to write a subclass of the abstract generators. Later I will show you how to write a subclass of RandomArray, etc.

2.5.1  SpecialString

Sometimes, we want to check special characters, for example the backslash ’' or unprinted characters ‘ESC’, ‘NUL’ and so on. SpecialString is a subclass of String which is defined in ‘rushcheck/string’. This library is already required in ‘rushcheck/rushcheck’ and don’t need to require again.

The output of random generator of SpecialString has different distribution of its of String. At String, the distribution of characters are normalized. On the other hand, the distribution in SpecialString is weighted and the generator provides special characters more often than alphabets. In detail, the distribution is as follows:

Table 1.  The distribution of SpecialString

Control characters50% (such as NUL)
Special characters25% (such as ’\’)

Using SpecialString in assertions, it is more likely to find the counter example. The following example is the famous bug which is called ‘malformed format bug’.

malformed_format_string =
  RushCheck::Assertion.new(String) { |s| sprintf(s); true }

malformed_format_string2 =
  RushCheck::Assertion.new(SpecialString) { |s| sprintf(s); true }

irb> malformed_format_string.check
Falsifiable, after 86 tests:
Unexpected exception: #<ArgumentError: malformed format string - %&>
        ... snip error traces ...
["\\n&'e!]hr(%&\\031Vi\\003 }ss"]
irb> malformed_format_string2.check
Falsifiable, after 15 tests:
Unexpected exception: #<ArgumentError: malformed format string>

In these results, we can see RushCheck find the counter example after 86 tests with String, on the other hand, find it quickly after 15 tests with SpecialString.

It is easy to change the distribution in SpecialString. You can define your subclass of SpecialString as follows:

class YourSpecialString < SpecialString
  @@frequency = { 'alphabet' => 1,
                  'control'  => 0,
                  'number'   => 0,
                  'special'  => 9 }

2.5.2  Array and RandomArray

The meaning of randomness for Array must be changed in testing codes. Sometimes you needs a random array of Integer, and another a random array of String. So what is random array?

RushCheck provides an abstract class RandomArray for abstract random generator. Programmer have to write a subclass of RandomArray as follows:

# for random array of integers
class MyRandomArray < RandomArray; end
MyRandomArray.set_pattern(Integer) {|ary, i| Integer }

The class method set_pattern takes a variable and a block. Because array is inductive structure, it can be defined by the basecase and the inductive step.

For example, let’s consider a random array in the following pattern [Integer, String, Integer, String, ...] where it has a random Integer at the odd index and a random String at the even index. Then we can write a random array with this pattern:

MyRandomArray.set_pattern(Integer) do |ary, i|
  if i % 2 == 0
  then Integer
  else String

More complecated example? OK, let’s consider a stream: * the first component is Integer * if the i-th component is positive integer * then (i+1)-th component is String * otherwise the (i+1)-th component is Integer

MyRandomArray.set_pattern(Integer) do |ary, i|
  if ary[i].kind_of?(Integer) && ary[i] >= 0
  then String
  else Integer

In this way, we can define any random array with any pattern.

2.5.3  Hash and RandomHash

Hash is also primitive and not so clear what is a random hash, like array. RushCheck provides an abstract random generator RandomHash, and programmer can write a subclass of RandomHash.

class MyRandomHash < RandomHash; end
pat = { 'key1' => Integer, 'key2' => String }

In this example, we can get a random Hash with two keys (‘key1’ and ‘key2’). Here the keys are String, but we can give any object as usual in Hash. Is it clear to define random hash? I think so. (If not it is clear, then the interface may be changed in future)

2.5.4  Proc and RandomProc

It is not difficult to create a random Proc object. As we saw in the previous sections, we have to write a subclass of RandomProc.

class MyRandomProc < RandomProc; end
MyRandomProc.set_pattern([Integer], [Integer])

Here, we define a random procedure which takes an integer and returns an integer also. In general, Ruby’s function and method can be regarded as a relation. (not a function in mathematics!) Therefore I think random procedures can be generated by a pair of inputs and outputs.

Let’s consider a simple example. In general, any functions f, g, and h should satisfy the associativity property:

bq. for all x, f(g(h(x)) === (f . g)(h (x)) where f . g is a composition of functions in mathematical way

class Proc
  # application
  def *(other)
    Proc.new do |*args|
      res = other.call(*args)

class MyRandomProc < RandomProc; end

def associativity_integer
  MyRandomProc.set_pattern([Integer], [Integer])
  RushCheck::Assertion.new(MyRandomProc, MyRandomProc,
                           MyRandomProc, Integer) do |f, g, h, x|
    (f * (g * h)).call(x) == ((f * g) * h).call(x)

P.S. The arbitrary method is used to create a random object for test instance. Then you may wonder what is the coarbitrary method? The coarbitrary method is used to generate a random procedure (lambda), which is one of central idea in QuickCheck.

2.6  Writing random generators

To use your class in the assertion, like

RushCheck::Assertion.new(YourClass) { |obj, ...| ... }
you have to write a code which generates a random object in your class. Therefore, at first we have to consider what is a random object in YourClass?

Sometimes the answer is not unique; there may be several ways for generating random objects. Like SpecialString in RushCheck, you can also write an abstract random generator.

OK, after thinking about the question, we have to write a code. To define random generators, we have to add a class method arbitrary in YourClass.

class YourClass
  extend RushCheck::Arbitrary

  def self.arbitrary
    # override the class method arbitrary

If you need to define a random proc which returns a object in YourClass, you have to include Coarbitrary, also.

class YourClass
  extend RushCheck::Arbitrary
  include RushCheck::Coarbitrary

  def self.arbitrary

  def coarbitrary(g)

However, because it is little complecated to implement both arbitrary and coarbitrary, let’s focus how to implement arbitrary first.

Let’s consider the first example: Candy. The Candy class requires its name and its price at initialization time. The name should be a String, and the price Integer. The Candy class may have several instance methods, but they are omitted because not important to define the random object.

class Candy

  def initialize(name, price)
    raise unless price >= 0
    @name, @price = name, price

  def foo

  def bar


To write random generator for Candy, we have to look up ‘initialize’. Here, assume that @name belongs String and @price belongs Integer. It is also natural that we assumes @price should be positive.

2.6.1  Using Gen.create

One simple random generator for Candy:

class Candy
  extend Arbitrary

  def self.arbitrary
    RushCheck::Gen.create(String, Integer) do |name, price|
      RushCheck::guard { price >= 0 }
      new(name, price)

Gen.create takes an array of Gen object (here,

) and a block. The block takes variables which is corresponded to the array, as Assertion.new. If guard is failed, then RushCheck retry to create another random instance.

Note that we can use a trick instead of the guard property.

price = - price if price < 0     # g.guard { price >= 0 }
In this case, this is more efficient to generate random instance than with the guard. However, sometimes we don’t have this kind trick and we can use some guards.

Note 1.  Changes in Gen.create for version ≥ 0.4


Remark: from version 0.4, Gen.create is changed to require classes in its argument from Gen objects.

Note 2.  Expensive candy


Using Gen.create to generate random instance, if we gives another guard such as

RushCheck::guard { price >= 100000 }    # very expensive candy!
then it consumes much time to generate random instance because the guard fails so many times.

When RushCheck creates random instances, it starts smallest as possible, then the size becomes larger in repeating tests. Therefore, if the guard instance seems failed so often, then we need another seeds of generators.

Here is another generators for expensive candy instance:

lo = 100000
g = RushCheck::Gen.sized { |n| RushCheck::Gen.choose(lo, n + lo) }
xs = [String.arbitrary, g]

See gen.rb and the following sections in details for how to get another generator.

2.6.2  Using Gen.bind and Gen.unit

class Candy
  extend RushCheck::Arbitrary

  def self.arbitrary
    String.arbitrary.bind do |name|
      Integer.arbitrary.bind do |price|
        price = - price if price < 0     # trick as I described above
        RushCheck::Gen.unit(new(name, price))

Puzzled? OK, they can be readed as follows:

  • take a random (arbitrary) string and call it ‘name’
  • take a random integer and call it ‘price’
  • return a Gen object which has a new Candy(name, price)

In general, the class method arbitrary should return a Gen object. Check also gen.rb in RushCheck. There are several combinators to create random generators.

There are several way to implement random generators. The next example is to use Gen.new without bind and unit.

2.6.3  Using Gen.new

class Candy
  extend RushCheck::Arbitrary

  def self.arbitrary
    RushCheck::Gen.new do |n, r|
      r2 = r
      name, price = [String, Integer].map do |c|
        r1, r2 = r2.split
        c.arbitrary.value(n. r1)
      price = - price if price < 0 # trick
      new(name, price)

This pattern is useful if your class has many valiables for initialize. Because binding patterns needs much depth of method call chains, it may be failed. This technique is used to avoid the stack overflow. This is one difference between Haskell and Ruby.

The implementation can be understanded as follows:

  • self.arbitrary returns a new Gen object.

    • let n be an integer and r be a random generator.

    • let r2 equal r

    • name and price are both random objects where

      • get new two random generator r1 and r2 from old r2

      • assign a random value by generating ‘arbitrary.value(n, r1)’

        • and discard the random generator r1

        • etc…

Note that we need new random generator for each random object. Here we create new random generator by spliting. If you use same random generator for generating different objects, then the distribution of objects are same (not generated randomly).

Because sometimes the initialize of YourClass is complecated, then the random generator self.arbitrary turns to be complicated also.

In next sections, I will introduce several generators in gen.rb. They may be useful to create your own random genrator. See also rdoc of Gen.

2.6.4  Other facilities in Gen class

To help defining random objects in your class, there are several functions in Gen class.  Gen.choose

Gen.choose(lo, hi) returns a Gen object which generates a random value in the bound.


Gen.choose(0, 10) # a generator of Integer in (0..10)  Gen.frequency

Gen.frequency requires an array of pair of Integer and Gen objects, and returns a Gen object. Gen.frequency is used to define the distribution of randomness.


# return a random integer or a random string (by
# choosing randomly) Integer:30% String: 70%

Gen.frequency([[3, Integer.arbitrary], [7, String.arbitrary]])

See also SpecialString.  Gen.lift_array

Gen.lift_array takes an array and a block which has a variable. The block should return a Gen object. lift_array returns a Gen object which generates an array of the result of given block for applying each member of given array.


class Candy
  extend RushCheck::Arbitrary

  def self.arbitrary
    RushCheck::Gen.lift_array([Integer, String]) do |c|
    end.bind do |args|
end  Gen.oneof

Gen.oneof requires an array of Gen objects returns a Gen object.


Gen.oneof([Integer.arbitrary, String.arbitrary])
  # return a random integer or a random string (by choosing
  # randomly)  Gen.promote

Gen.promote is used to generate a random Proc object. Next section I will describe how to define the random Proc object. See also proc.rb in the example directory of RushCheck.  Gen.rand

Gen.rand is a random number generator.  Gen.sized

Gen.sized is used to change the size of random object. See also some implementation in RushCheck (grep the source!)  Gen.unit

Returns a Gen object.  Gen.vector

Get a vector of Gen.


Gen.vector(Integer, 3) # => Gen [Integer, Integer, Integer]

2.7  Writing random procs

how to write random Proc which returns objects in YourClass

It is complecated, and see some examples.

  • rushcheck/bool.rb
  • rushcheck/integer.rb
  • rushcheck/string.rb

so on, grep ‘coarbitrary’

FIXME: how to write coarbitrary

Chapter 3
Further information

Webpage should have another useful information:

The project page has a bug tracker and webboard.

There is no mailing list for RushCheck (now at 2006-08-08), but don’t hesitate to contact to the author!

Happy hacking and testing!

This document was generated by ERBook 7.1.1 on Mon Sep 07 16:47:07 -0700 2009 using the following resources.

Resource Origin License
here_frag caution nav_here nav_prev tip important nav_list note warning quote nav_next Tango Icon Theme

© 2005 Tango Desktop Project

Creative Commons Attribution-ShareAlike 2.5 License Agreement
hyperlink MediaWiki Monobook Skin

© 2007 MediaWiki contributors

GNU General Public License, version 2

Valid XHTML 1.0 Transitional Valid CSS!