Byte code execution count

Kenneth P. Turvey

I'm thinking about working on a project with a friend of mine, but there
are some issues that I'm not sure how to handle. I'm going to break them
into a few different posts since the issues are unrelated to each other
and people may choose to ignore some of them that way.

I understand that Java doesn't allow one to find out how much CPU time was
used by a process. That's fine. This isn't really a good measure for
what we want anyway since it would be vastly different from CPU to CPU.
What we really want is a way to measure how much work was done by a given
CPU on a given job.

What I was thinking would be a good measure was a simple count of the java
byte code instructions executed on a given machine. Is there anyway to
determine how many byte code instructions have been executed by a given
thread since it started?

I'm sure there is a way to do this with the debugging information, but I
don't want to slow down the process too much. Basically I'd like to be
able to start a thread and then wait until it exits and then ask how many
byte code instructions it processed? Any idea how one would do this?

Any other ideas on how to solve this sort of problem?

Thanks.



--
Kenneth P. Turvey <kt-usenet@squeakydolphin.com>

Eric Sosman

Kenneth P. Turvey wrote On 10/23/07 11:25,:
> I'm thinking about working on a project with a friend of mine, but there
> are some issues that I'm not sure how to handle. I'm going to break them
> into a few different posts since the issues are unrelated to each other
> and people may choose to ignore some of them that way.
>
> I understand that Java doesn't allow one to find out how much CPU time was
> used by a process. That's fine. This isn't really a good measure for
> what we want anyway since it would be vastly different from CPU to CPU.
> What we really want is a way to measure how much work was done by a given
> CPU on a given job.
>
> What I was thinking would be a good measure was a simple count of the java
> byte code instructions executed on a given machine. Is there anyway to
> determine how many byte code instructions have been executed by a given
> thread since it started?
>
> I'm sure there is a way to do this with the debugging information, but I
> don't want to slow down the process too much. Basically I'd like to be
> able to start a thread and then wait until it exits and then ask how many
> byte code instructions it processed? Any idea how one would do this?
>
> Any other ideas on how to solve this sort of problem?

I think you're out of luck, and for more than one
reason.

First, I can think of no way to discover which of
the machine's CPU's ran which parts of your Java code.
That's difficult to do even in assembly, let alone in
Java.

Second, bytecode count is unlikely to be a very
accurate "progress" measure. Bytecode instructions
like iconst_0 are very simple, take little time, and
make little "progress." Instructions like athrow do
considerably more work, take more time, and make more
"progress" (although perhaps in reverse ...).

Finally, ask yourself how many bytecodes a method
executes after it's been JITted ...

"Progress" is really an application-specific notion,
not a machine-provided commodity. The machine can only
run the program; it can't tell whether the program is
"doing something" or just sitting in a polling loop
waiting for something else. If you want to keep track
of "progress," I think you need to establish milestones
within the application itself and count them as they pass.

--
Eric.Sosman@sun.com

Kenneth P. Turvey

On Tue, 23 Oct 2007 12:31:38 -0400, Eric Sosman wrote:

> "Progress" is really an application-specific notion,
> not a machine-provided commodity. The machine can only
> run the program; it can't tell whether the program is
> "doing something" or just sitting in a polling loop
> waiting for something else. If you want to keep track
> of "progress," I think you need to establish milestones
> within the application itself and count them as they pass.

I guess I'm not really interested in progress as such.. more like expense.

I don't really care if the client code is spinning in a loop doing
nothing. If it is wasting my CPU time it counts. So what I'm really
trying to measure here is a sort of normalized CPU time. Ideally,
something like how many CPU seconds this command would spend on my
Inspiron 1420N would be a terrific measure.

I know that might be a bit too much to ask, so I'm looking for something I
can get. I understand your point though. The JIT compiler really would
completely mess this statistic up.

Any other ideas?

If the numbers came out +/- 30% that would probably be fine. I'm just
trying to get a rough measure.



--
Kenneth P. Turvey <kt-usenet@squeakydolphin.com>

Roedy Green

On 23 Oct 2007 15:25:03 GMT, "Kenneth P. Turvey"
<kt-usenet@squeakydolphin.com> wrote, quoted or indirectly quoted
someone who said :

>
>What I was thinking would be a good measure was a simple count of the java
>byte code instructions executed on a given machine. Is there anyway to
>determine how many byte code instructions have been executed by a given
>thread since it started?

not in hotspot, because the byte codes are not used. They are
converted to native code. You can get a nanotimer tick. See
http://mindprod.com/jgloss/time.html

Of course it also counts time used by other tasks too.
--
Roedy Green Canadian Mind Products
The Java Glossary
http://mindprod.com

Lew

Kenneth P. Turvey wrote:
> If the numbers came out +/- 30% that would probably be fine. I'm just
> trying to get a rough measure.

Simply switching the JVM from -client to -server can speed it up 50%, that is,
the process would take half the time.

I've seen this happen, measured on an IBM JVM in a large enterprise system,
running on a multi-CPU system.

The optimizer can really change things, not just the time code takes, but the
space through inlining.

You need to find a different metric. Aren't any of the profilers out there
useful to you?

--
Lew

Kenneth P. Turvey

On Tue, 23 Oct 2007 17:41:50 -0400, Lew wrote:

> Kenneth P. Turvey wrote:
>> If the numbers came out +/- 30% that would probably be fine. I'm just
>> trying to get a rough measure.
>
> Simply switching the JVM from -client to -server can speed it up 50%, that is,
> the process would take half the time.
>
> I've seen this happen, measured on an IBM JVM in a large enterprise system,
> running on a multi-CPU system.
>
> The optimizer can really change things, not just the time code takes, but the
> space through inlining.
>
> You need to find a different metric. Aren't any of the profilers out there
> useful to you?

Hmm... what I'm looking for is a metric that would be useful when
averaged over 1000 runs on 1000 different computers. I'd like to be able
to compare two different programs and be able to say with some sense of
confidence that program A requires twice as many resources as program B
after this exercise (of course only referring to CPU, not other resources).

A profiler doesn't help because this will all be happening on remote
machines.



--
Kenneth P. Turvey <kt-usenet@squeakydolphin.com>

Patricia Shanahan

Kenneth P. Turvey wrote:
> On Tue, 23 Oct 2007 17:41:50 -0400, Lew wrote:
>
>> Kenneth P. Turvey wrote:
>>> If the numbers came out +/- 30% that would probably be fine. I'm just
>>> trying to get a rough measure.
>> Simply switching the JVM from -client to -server can speed it up 50%, that is,
>> the process would take half the time.
>>
>> I've seen this happen, measured on an IBM JVM in a large enterprise system,
>> running on a multi-CPU system.
>>
>> The optimizer can really change things, not just the time code takes, but the
>> space through inlining.
>>
>> You need to find a different metric. Aren't any of the profilers out there
>> useful to you?
>
> Hmm... what I'm looking for is a metric that would be useful when
> averaged over 1000 runs on 1000 different computers. I'd like to be able
> to compare two different programs and be able to say with some sense of
> confidence that program A requires twice as many resources as program B
> after this exercise (of course only referring to CPU, not other resources).
>
> A profiler doesn't help because this will all be happening on remote
> machines.

Have you looked at java.lang.management? I use it to collect data like
the total CPU time for a job.

Patricia

Lew

Lew wrote:
>> Simply switching the JVM from -client to -server can speed it up 50%, that is,
>> the process would take half the time.

Kenneth P. Turvey wrote:
> Hmm... what I'm looking for is a metric that would be useful when
> averaged over 1000 runs on 1000 different computers. I'd like to be able
> to compare two different programs and be able to say with some sense of
> confidence that program A requires twice as many resources as program B
> after this exercise (of course only referring to CPU, not other resources).

In the case I observed, the very same program running on the very same machine
required twice as much CPU in one configuration as in another. Would you
consider this program to be more compact on one configuration than the other,
even though it's the very same bytecode?

--
Lew

Kenneth P. Turvey

On Tue, 23 Oct 2007 16:11:50 -0700, Patricia Shanahan wrote:

> Have you looked at java.lang.management? I use it to collect data like
> the total CPU time for a job.

That will probably work. I would have to average it over all the
machines, but it might be good enough. Byte codes would be better, but I
guess you can't always get exactly what you want.

If I pick 100 machines at random and run some code on them and time it and
then, a week later, pick another 100 machines at random and run the same
code on them, do you think I'll come up with close to the same number?

I'll have to think about the statistics involved in that.

Thanks.

--
Kenneth P. Turvey <kt-usenet@squeakydolphin.com>

Lew

Kenneth P. Turvey wrote:
> On Tue, 23 Oct 2007 16:11:50 -0700, Patricia Shanahan wrote:
>
>> Have you looked at java.lang.management? I use it to collect data like
>> the total CPU time for a job.
>
> That will probably work. I would have to average it over all the
> machines, but it might be good enough. Byte codes would be better, but I
> guess you can't always get exactly what you want.

But, bytecodes cannot be consistently counted, sometimes not even at all, as
others have pointed out on this thread. How would they be better?

--
Lew