codeflood logo

Generate Random Test Data

I had an instance recently where I needed to generate a large volume of test data. In this particular case, it was hundreds of thousands of extranet users. This sounds like a job for Revolver! Unfortunatley the current version of Revolver lacks control and repition constructs; the things that allow the generation of such large volumes of data. Luckily, I worked out how to create some custom commands to execute within Reolver to achieve the desired outcome.

First, we need a repition command that will execute a command passed to it a number of times. Next we'll need a random data generator command to generate the random data.

Repeat Command

We'll start with the repition command. This command should accept 2 parameters. The first is the number of times to execute, and the second should be the command we want to execute. The signature of the command should look like the following:

 100 command

The following code will achieve this. Wack this into a VS project and compile it.

using System;
using System.Text;
using Revolver.Core;

namespace CustCommand
{
	public class Repetition : ICommand
	{
		private Context m_context = null;
		private IFormatContext m_formatContext = null;

		public string Description()
		{
			return "Allows repititous execution of a command";
		}

		public HelpDetails Help()
		{
			HelpDetails details = new HelpDetails();
			details.Usage = "<cmd> repetitions command";
			details.AddParameter("repetitions", "The number of repititions to run");
			details.AddParameter("command", "The command to run");
			details.AddExample("<cmd> 5 pwd");
			details.AddExample("<cmd> 2 (cd ..)");
			return details;
		}

		public void Initialise(Context context, IFormatContext formatContext)
		{
			m_context = context;
			m_formatContext = formatContext;
		}

		public CommandResult Run(string\[\] args)
		{
			if (args.Length != 2)
				return new CommandResult(CommandStatus.Failure, "Wrong number of parameters");

			int reps = 0;
			int.TryParse(args\[0\], out reps);

			string command = args\[1\];
			StringBuilder sb = new StringBuilder();

			for (int i = 0; i < reps; i++)
			{
				sb.Append(m\_context.ExecuteCommand(command, m\_formatContext));
			}

			return new CommandResult(CommandStatus.Success, sb.ToString());
		}
	}
}

Random Data Generator

Next we need to generate some random character data. The command should accept a number to determine the number of random characters to output. The signature would look like the following:

 10

The following code will do this.

using System;
using System.Text;
using Revolver.Core;

namespace CustCommand
{
	public class Random : ICommand
	{
		private Context m_context = null;
		private IFormatContext m_formatContext = null;		

		public string Description()
		{
			return "Generates random characters";
		}

		public HelpDetails Help()
		{
			HelpDetails details = new HelpDetails();
			details.Usage = "<cmd> num";
			details.AddParameter("num", "The number of characters to return");
			details.AddExample("<cmd> 7");
			return details;
		}

		public void Initialise(Context context, IFormatContext formatContext)
		{
			m_context = context;
			m_formatContext = formatContext;
		}

		public CommandResult Run(string\[\] args)
		{
			if (args.Length != 1)
				return new CommandResult(CommandStatus.Failure, "Wrong number of parameters");

			int num = 0;
			int.TryParse(args\[0\], out num);
			StringBuilder sb = new StringBuilder(num);
			string chars = "abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ1234567890";
			System.Random rand = new System.Random();

			for (int i = 0; i < num; i++)
			{
				sb.Append(chars\[rand.Next(1, chars.Length) - 1\]);
			}

			return new CommandResult(CommandStatus.Success, sb.ToString());
		}
	}
}

Deploy and Run

Both classes should be compiled into a class library assembly, then dropped into the bin folder of your Sitecore website. Now we need to bind the custom commands into the Revolver shell.

bind CustCommand.Random,CustCommand rand
bind CustCommand.Repetition,CustCommand rep

Now that we have the desired commands, we need to construct a new command to generate some random test items. The following command will generate 10 random items, based on the document template in the current location, each with a name of 5 characters in length.

rep 10 (touch -t Sample/Document < (rand 5))

From this point it's quite easy to create a script based on the above command to generate our random data. We could even take it a step further and fill in fields of the items once created. The following script accepts 2 parameters. The first is the number of items to create, the second is the template to base the new items on. Note that we bind the custom commands in at the start of the script to ensure they are loaded when required.

bind CustCommand.Repetition,CustCommand rep
bind CustCommand.Random,CustCommand rand
rep $1$ (touch -t $2$ < (rand 5))

To be able to fill in the fields as well, we need to seperate the item name generation from the item creation, so we can use the name as a path. The rep custom command above only accepts a single parameter for the command to execute, so we need to first create a script which we can pass to rep which will create the item given the template, then fill in the fields with random data. Take the below script and put it into a script called genitem.

bind CustCommand.Random,CustCommand rand
set name < (rand 5)
touch -t $1$ $name$
cd $name$
sf title < (echo title < (rand 3))
sf text < (echo text < (rand 200))
cd ..

This script can now be passed into the rep custom commnad.

rep 10 (genitem Sample/Document)

When there are too many items in a single location in Sitecore performance can degrade. I saw a post on Alex de Groot's blog recently titled The maximum of Sitecore items? At least a million! which spoke about the maximum recommended child items for any location to be between 100 to 200 items. So if we want thousands of nodes we have to make good use of folders. And if we want to generate folders as part of the test data? Well we could incorporate that into our scrips.

Let's create a set of scripts to take care of creating a structure of folders. We'll create a heirarchy of folders 3 levels deep each with a set of items in it. This solution is a little ugly as we don't have any control constructs, but it will work. Once the first level of folders is created, we want to step into each and generate more folders and more items. Again, at the second level we want to step into it and create the 3rd level. We could use some recursion (scripts calls itself) if we had control constructs like if or while, but alas we do not. Instead, we'll create a seperate script for each level, but they will almost be the same. The first level script only has to repeatedly call the next level script to generate the ifrst level of folders and items, so let's do that. Note that I'll make a call in this script to the next script we'll create called genlevel2.

bind CustCommand.Repetition,CustCommand rep
bind CustCommand.Random,CustCommand rand
rep 10 genlevel2

The genlevel2 script doesn't have to worry about binding in the custom commands, as we intend for this script to be called from the first script which has already bound the commands. In this script, we'll generate the first level folders, kick of the script for the next level and create the sample items for the first level folder using our previously created genitem script from above.

set f1 < (rand 5)
touch -m folder $f1$
cd $f1$
rep 10 genlevel3
cd ..
rep 10 (genitem sample/document)

genlevel3 will be the same as genlevel2 as it also needs to create a new level of folders and some test items. We just need to change the script which it calls recursively.

set f1 < (rand 5)
touch -m folder $f1$
cd $f1$
rep 10 genlevel4
cd ..
rep 10 (genitem sample/document)

The next script, genlevel4 will be exactly the same as genlevel3 except that we need to remove the command to create another level of folders.

set f1 < (rand 5)
touch -m folder $f1$
cd $f1$
rep 10 (genitem sample/document)
cd ..

After executing the first script we'll have 10^1 + 10^2 + 10^3 = 1,110 folders and 10 * 1,110 folders = 11,100 test items nested within the folders. That's 12,210 items. That's a nice amount of test data to test any component! And of course you could modify these scripts to allow the passing of the number of items to create as a parameter.