So I create a very simple XMLRPC server in Python by gluing together the necessary parts from built-in and third-party modules and create a simple VB client that uses MSXML to call it. Then I set up a loop to call an Oracle stored procedure via my python server 1000 times, and a similar loop to call the exact same stored procedure 1000 times via our ActiveX DLL in COM+. The XMLRPC loop took one third as long as the COM+ DLL loop.

Then I ran three instances of the looping program at the same time and the python loops took about three times as long as the original run, but the COM+ loops were still running after ten minutes.

I went to our lead developer to brag about python smoking our COM+ DLL in trivial scalability tests and now I am assigned to figuring out why the COM+ DLL becomes such a bottleneck and trying to fix it. OY.

Here's a way to fix it... switch to python.


infidel

$ select * from users where clue > 0
no rows returned

: I went to our lead developer to brag about python smoking our COM+ DLL in trivial scalability tests and now I am assigned to figuring out why the COM+ DLL becomes such a bottleneck and trying to fix it. OY.
:

Tell them the problem is inherent to the way COM+ is done and it can't be fixed. Sort of like an ASCII file can't contain Chr$(0). You can't "fix" it; if you manually add a Chr$(0), it's not an ASCII file anymore.

Of course, I don't know this but I'm sure I can draw up... I mean, find, yeah that's it, a document indicating such. <grin>

: : I went to our lead developer to brag about python smoking our COM+ DLL in trivial scalability tests and now I am assigned to figuring out why the COM+ DLL becomes such a bottleneck and trying to fix it. OY.
: :
:
: Tell them the problem is inherent to the way COM+ is done and it can't be fixed. Sort of like an ASCII file can't contain Chr$(0). You can't "fix" it; if you manually add a Chr$(0), it's not an ASCII file anymore.
:
: Of course, I don't know this but I'm sure I can draw up... I mean, find, yeah that's it, a document indicating such. <grin>
:

I'm guessing the COM+ infrastrucrure is adding another layer of code execution. I would think especially if the component was commiting and rolling back transactions.

: I'm guessing the COM+ infrastrucrure is adding another layer of code execution. I would think especially if the component was commiting and rolling back transactions.

That's the interesting thing. There's about the same number of layers.

Python way:
1. VB program creates valid XMLRPC structure.
2. VB program instantiates XHTTP object from MSXML
3. VB program calls socket server written in Python on same machine
4. Python program unpacks XMLRPC request
5. Python program calls stored procedure using Oracle module
6. Python program packs result into XMLRPC response
7. Python program sends response back to XHTTP client and closes socket

COM+ way:
1. VB program instantiates ActiveX DLL class
2. VB program calls method on new object
3. Object uses ADO recordset to pack information into an XML structure
4. Object instantiates ActiveX DLL class registered with COM+
5. Object passes XML structure to COM+ class
6. COM+ class unpacks ADO XML information
7. COM+ class calls stored procedure using ADO and OLEDB provider
8. COM+ class passes stored procedure return value (string) back to first object

You would think that using ActiveX DLLs would be much faster than XMLRPC and a python script. In fact, if the VB program instantiates ADO objects itself rather than using our database proxy class(es), it runs in about half the time as the python way. So the problem isn't ADO per se, but most likely in the class(es) we have running under COM+. COM+ is used specifically for scalability and robustness. Which becomes all the more amazing when running multiple instances of my VB client causes the COM+ class' performance to drop off by orders of magnitude.

Today I am going to get a log of the network traffic between my machine and the Oracle database. This should allow me to eliminate the database access as a factor in the differences. My python server is using a SessionPool and COM+ is supposed to be pooling connections, so connection time shouldn't be a factor.


infidel

$ select * from users where clue > 0
no rows returned

: : I'm guessing the COM+ infrastrucrure is adding another layer of code execution. I would think especially if the component was commiting and rolling back transactions.
:
: That's the interesting thing. There's about the same number of layers.
:
: Python way:
: 1. VB program creates valid XMLRPC structure.
: 2. VB program instantiates XHTTP object from MSXML
: 3. VB program calls socket server written in Python on same machine
: 4. Python program unpacks XMLRPC request
: 5. Python program calls stored procedure using Oracle module
: 6. Python program packs result into XMLRPC response
: 7. Python program sends response back to XHTTP client and closes socket
:
: COM+ way:
: 1. VB program instantiates ActiveX DLL class
: 2. VB program calls method on new object
: 3. Object uses ADO recordset to pack information into an XML structure
: 4. Object instantiates ActiveX DLL class registered with COM+
: 5. Object passes XML structure to COM+ class
: 6. COM+ class unpacks ADO XML information
: 7. COM+ class calls stored procedure using ADO and OLEDB provider
: 8. COM+ class passes stored procedure return value (string) back to first object
:
: You would think that using ActiveX DLLs would be much faster than XMLRPC and a python script. In fact, if the VB program instantiates ADO objects itself rather than using our database proxy class(es), it runs in about half the time as the python way. So the problem isn't ADO per se, but most likely in the class(es) we have running under COM+. COM+ is used specifically for scalability and robustness. Which becomes all the more amazing when running multiple instances of my VB client causes the COM+ class' performance to drop off by orders of magnitude.
:
: Today I am going to get a log of the network traffic between my machine and the Oracle database. This should allow me to eliminate the database access as a factor in the differences. My python server is using a SessionPool and COM+ is supposed to be pooling connections, so connection time shouldn't be a factor.
:
:
: infidel
:
:

: $ select * from users where clue > 0
: no rows returned
: 

:
:

COM+ is another layer on top of the base COM API ... right?

VB is a COM pig, because it has to implement extra interfaces behind the scenes to be compatible. Some Active-x controls, ive heard, implement between 7 and 15 interfaces.

Also think about VB object creation ...

A. All objects implement IUnknown
B. Query IUnknown for such and such ...
C. Maintain reference count behind the scenes ...

COM+ expands COM ... COM+ hijacks your vb component and adds more invisible code execution ... since COM+ requires components to be stateless, your component is creating and destroyin objects with every call i think ...

Also, ive had problems w/ COM+ and it's one of those frameworks you have to use absolutely correct, or you will suffer.

Now weigh that with the fact that Python isn't COM and doesn't have as much leg work to get an object reference. If you think about it, it calls upon behind-the-scenes C or C++ like VB but without the COM.

These are all theories that i'm clawing at, so tell me what u think.

Also, if u didn't do this already, and i'm sure u did ... compile the vb dll with all the compiler oprimizations like ... "Turning off bounds checking" etc...

I hear you can sometimes increase the speed of the vb code two-fold.

Also try the bench-mark without COM+.

This message was edited by infidel at 2004-1-29 13:32:31

---

: COM+ is another layer on top of the base COM API ... right?

In some sense of the word, I think so.

: VB is a COM pig, because it has to implement extra interfaces behind the scenes to be compatible. Some Active-x controls, ive heard, implement between 7 and 15 interfaces.

Yeah, but it is compiled, compared to my python server which is interpreted.

: Also think about VB object creation ...
:
: A. All objects implement IUnknown
: B. Query IUnknown for such and such ...
: C. Maintain reference count behind the scenes ...

I've got to believe that this doesn't add all that much overhead.

: COM+ expands COM ... COM+ hijacks your vb component and adds more invisible code execution ... since COM+ requires components to be stateless, your component is creating and destroyin objects with every call i think ...

Essentially. But the thing COM+ is supposed to do is add scalability by reducing the time it takes to create objects. DLLHOST.exe keeps running for a specific period of time after the refcount drops to zero in case a new instance needs to be created.

: Also, ive had problems w/ COM+ and it's one of those frameworks you have to use absolutely correct, or you will suffer.

Yeah, our lead developer had had major problems with it, but insists it's a necessary evil for scalability.

: Now weigh that with the fact that Python isn't COM and doesn't have as much leg work to get an object reference. If you think about it, it calls upon behind-the-scenes C or C++ like VB but without the COM.

Well, somewhat. The base async socket classes in the standard library are implemented in C, but my subclasses of those classes are not, so there's still some dynamic python name lookup stuff going on.

: These are all theories that i'm clawing at, so tell me what u think.

All very interesting.

: Also, if u didn't do this already, and i'm sure u did ... compile the vb dll with all the compiler oprimizations like ... "Turning off bounds checking" etc...

I didn't think of that yet. I'm still trying to get a sense of the problem. Been distracted today with some emergency data fixes/recovery.

: I hear you can sometimes increase the speed of the vb code two-fold.

That would be interesting.

: Also try the bench-mark without COM+.

That would require some extra work in a separate project as the DLL references and calls some stuff from the COM+ library to determine the original caller's userid. It won't run as-is outside of COM+.

Here's something interesting...

If I open the COM+ control panel thing and watch the components reference counts and call times while the loop is running, something surprising happens. If I have one instance of the loop running, the call time averages about 20-25 milliseconds. With two loops running concurrently the call time increases 10-20 times that. With three loops running it ranges anywhere from 30 to 2500 milliseconds, with the majority in the 1000 - 1100 range. This all seems to indicate to me a lack of scalability. I mean, with three loops running there's a total of 3000 calls being made through the DLL to the database and it bogs down like a mofo.


infidel

$ select * from users where clue > 0
no rows returned

: : COM+ is another layer on top of the base COM API ... right?
:
: In some sense of the word, I think so.
:
I'm no expert, but that's what I've heard too.

: : Now weigh that with the fact that Python isn't COM and doesn't have as much leg work to get an object reference. If you think about it, it calls upon behind-the-scenes C or C++ like VB but without the COM.
:
: Well, somewhat. The base async socket classes in the standard library are implemented in C, but my subclasses of those classes are not, so there's still some dynamic python name lookup stuff going on.
:
: : These are all theories that i'm clawing at, so tell me what u think.
:
: All very interesting.
:
: : Also, if u didn't do this already, and i'm sure u did ... compile the vb dll with all the compiler oprimizations like ... "Turning off bounds checking" etc...
:
: I didn't think of that yet. I'm still trying to get a sense of the problem. Been distracted today with some emergency data fixes/recovery.
:
: : I hear you can sometimes increase the speed of the vb code two-fold.
:
: That would be interesting.
:
Depends on what you're doing. When the AMaMP mix engine was written in VB I saw *big* performance gains (very possibly two-fold) by turning on the optimizations. Now it's written in C and a debug build is still up to 10 times faster than the VB original. It also compiles on other OSes. But that aside, I'm not convinced you'd make such gains with the optimisations in your average VB business logic.

: : Also try the bench-mark without COM+.
:
: That would require some extra work in a separate project as the DLL references and calls some stuff from the COM+ library to determine the original caller's userid. It won't run as-is outside of COM+.
:
: Here's something interesting...
:
: If I open the COM+ control panel thing and watch the components reference counts and call times while the loop is running, something surprising happens. If I have one instance of the loop running, the call time averages about 20-25 milliseconds. With two loops running concurrently the call time increases 10-20 times that. With three loops running it ranges anywhere from 30 to 2500 milliseconds, with the majority in the 1000 - 1100 range. This all seems to indicate to me a lack of scalability. I mean, with three loops running there's a total of 3000 calls being made through the DLL to the database and it bogs down like a mofo.
:
This kinda growth almost looks exponential. Which is very bad. My guess is that it's to do with resource locking or something similar. Is it to do with COM+, or is there something on the database side at a connection level (I'm assuming that each COM+ instance has it's own connection). I can imagine a situation where one instance is hanging onto a connection (for pooling reasons), and thus a lock of some kind, and until it releases it the other instance can't do it's call.

On the other hand, I could be completely wrong and obviously way off base because this isn't really my area.

Jonathan

###
for(74,117,115,116){$::a.=chr};(($_.='qwertyui')&&
(tr/yuiqwert/her anot/))for($::b);for($::c){$_.=$^X;
/(p.{2}l)/;$_=$1}$::b=~/(..)$/;print("$::a$::b $::c hack$1.");

: : : COM+ is another layer on top of the base COM API ... right?
: :
: : In some sense of the word, I think so.
: :
: I'm no expert, but that's what I've heard too.

I think COM+ is kind of a "framework" that uses COM interfaces to provide a "robust, scalable middle tier". Or something like that.

: This kinda growth almost looks exponential.

That's why I was so stunned.

: Which is very bad.

Yup.

: My guess is that it's to do with resource locking or something similar.

It seems now that threading is a likely culprit. VB can only compile components to use two different threading models. "Single" or "Apartment". Turns out the "Apartment" style in VB equates to the "Single-Threaded-Apartment" or STA. There are also Multithreaded and Neutral models as well, but they're not available from VB. From what I've read, STA makes code much simpler to write because COM+ handles all of the marshalling and locking blah blah blah. It comes, however, with a relatively significant performance bottleneck because an object instance can only be activated by the thread that created it (it's apartment), meaning COM+ has to juggle requests between the RPC thread that received a request and the thread that owns the object instance. This seems a reasonable explanation, as more and more users are calling on our component at the same time, COM+ has more and more threads to synchronize, etc.

: Is it to do with COM+, or is there something on the database side at a connection level (I'm assuming that each COM+ instance has it's own connection). I can imagine a situation where one instance is hanging onto a connection (for pooling reasons), and thus a lock of some kind, and until it releases it the other instance can't do it's call.

I'm pretty sure each instance has it's own connection, but our Oracle database can handle massive amounts of connections so I don't think that is an issue.

: On the other hand, I could be completely wrong and obviously way off base because this isn't really my area.

It's now my area, for the moment at least.


infidel

$ select * from users where clue > 0
no rows returned

: : : : COM+ is another layer on top of the base COM API ... right?
: : :
: : : In some sense of the word, I think so.
: : :
: : I'm no expert, but that's what I've heard too.
:
: I think COM+ is kind of a "framework" that uses COM interfaces to provide a "robust, scalable middle tier". Or something like that.
:
: : This kinda growth almost looks exponential.
:
: That's why I was so stunned.
:
: : Which is very bad.
:
: Yup.
:
: : My guess is that it's to do with resource locking or something similar.
:
: It seems now that threading is a likely culprit. VB can only compile components to use two different threading models. "Single" or "Apartment". Turns out the "Apartment" style in VB equates to the "Single-Threaded-Apartment" or STA. There are also Multithreaded and Neutral models as well, but they're not available from VB. From what I've read, STA makes code much simpler to write because COM+ handles all of the marshalling and locking blah blah blah. It comes, however, with a relatively significant performance bottleneck because an object instance can only be activated by the thread that created it (it's apartment), meaning COM+ has to juggle requests between the RPC thread that received a request and the thread that owns the object instance. This seems a reasonable explanation, as more and more users are calling on our component at the same time, COM+ has more and more threads to synchronize, etc.
:
: : Is it to do with COM+, or is there something on the database side at a connection level (I'm assuming that each COM+ instance has it's own connection). I can imagine a situation where one instance is hanging onto a connection (for pooling reasons), and thus a lock of some kind, and until it releases it the other instance can't do it's call.
:
: I'm pretty sure each instance has it's own connection, but our Oracle database can handle massive amounts of connections so I don't think that is an issue.
:
: : On the other hand, I could be completely wrong and obviously way off base because this isn't really my area.
:
: It's now my area, for the moment at least.
:
:
: infidel
:
:

: $ select * from users where clue > 0
: no rows returned
: 

:
:

With Active-X EXE i think, VB6 gives you the option to increase the number of threads ?!? Been awhile.

: With Active-X EXE i think, VB6 gives you the option to increase the number of threads ?!? Been awhile.

I'm not sure but I think COM+ only works with DLLs.


infidel

$ select * from users where clue > 0
no rows returned

: It seems now that threading is a likely culprit. VB can only compile components to use two different threading models. "Single" or "Apartment". Turns out the "Apartment" style in VB equates to the "Single-Threaded-Apartment" or STA. There are also Multithreaded and Neutral models as well, but they're not available from VB. From what I've read, STA makes code much simpler to write because COM+ handles all of the marshalling and locking blah blah blah. It comes, however, with a relatively significant performance bottleneck because an object instance can only be activated by the thread that created it (it's apartment), meaning COM+ has to juggle requests between the RPC thread that received a request and the thread that owns the object instance. This seems a reasonable explanation, as more and more users are calling on our component at the same time, COM+ has more and more threads to synchronize, etc.

I may have a chance here to re-write our component in C++ to take advantage of the more scalable threading models available. Wish me luck.


infidel

$ select * from users where clue > 0
no rows returned

Good luck w/ the C++, we talking the danger zone.

: Good luck w/ the C++, we talking the danger zone.

Amen to that. I wish I could come up with a solid python solution, but I'm still in the tinkering phase there. I've given up using the asyncore module directly to try out the twisted framework that provides wrappers for all standard protocols around an asynchronous event loop. It still smokes the COM+ way, but only as long as the calls are very short. Anyways, at least I've got some excuses to be playing around with different languages for a while.


infidel

$ select * from users where clue > 0
no rows returned

This message was edited by iwilld0it at 2004-1-29 19:46:43

---

This message was edited by iwilld0it at 2004-1-29 19:42:58

---

What are you doing in these loops roughly? I know a fair amount of ADO and I know that scrolling through recordsets other-than forward-only and read-only, you looking at slower loops . Also, you are looking at performance differences depending on where the cursors are created (server or client.) Are you using OLEDB or ODBC to connect? Are you issuing Database commands per loop?

I find it more preferable to issue a batch of commands all at once ... outside of loops ...

Dim SQL As String
SQL = "EXEC MySproc 'A';EXEC MySproc 'B';EXEC MySproc 'C'"
' pass sql to command

Or call a stored procedure that creates a cursor to offload that looping mechanism onto Oracle.

Judging by your observations, the code seems to be running at O^3 complexity (Which backs John's exponential growth theory.)

Definitely a bottle-neck somewhere. Maybe too many connections opening without closing ... I dunno ... with ADO, alot of factors can reduce scalability and COM+ is probably doing it's job correct but, is not god enough to fix it.

If your using an NT based OS, look at performance counters under ADO to see what is going on. Also, I know SQL-Server has performance monitor for analyzing to see how many connections are opened and closed. Maybe Oracle has a similar utility???

: This message was edited by iwilld0it at 2004-1-29 19:46:43

---

: This message was edited by iwilld0it at 2004-1-29 19:42:58

---

: What are you doing in these loops roughly?

Getting the name of the database.

: I know a fair amount of ADO and I know that scrolling through recordsets other-than forward-only and read-only, you looking at slower loops . Also, you are looking at performance differences depending on where the cursors are created (server or client.)

Perfectly valid insights, but no cursors are involved here.

: Are you using OLEDB or ODBC to connect?

OLEDB using the MSDAORA provider.

: Are you issuing Database commands per loop?

Just one stored procedure call that takes no parameters and returns one varchar2

: I find it more preferable to issue a batch of commands all at once ... outside of loops ...
: Or call a stored procedure that creates a cursor to offload that looping mechanism onto Oracle.

Oh definitely. I learned a long time ago to make Oracle do the work when there's a lot of processing to do. It can finish the job while VB is still gathering everything it needs to begin.

: Judging by your observations, the code seems to be running at O^3 complexity (Which backs John's exponential growth theory.)

All VB does is call a stored procedure that returns a string. All the stored procedure does is:

select global_name into v_return from global_name

Then it does a substr() to chop off the domain and only return the instance's SID.

: Definitely a bottle-neck somewhere. Maybe too many connections opening without closing ... I dunno ... with ADO, alot of factors can reduce scalability and COM+ is probably doing it's job correct but, is not god enough to fix it.

If I use straight ADO objects then everything goes even faster than my python xmlrpc server. There definitely seems to be something "wrong" with our component that wraps the calls to ADO. At least as far as COM+ scalability goes.

: If your using an NT based OS, look at performance counters under ADO to see what is going on. Also, I know SQL-Server has performance monitor for analyzing to see how many connections are opened and closed. Maybe Oracle has a similar utility???

We've got a number of options. If anything interesting happens I'll be sure to report back.


infidel

$ select * from users where clue > 0
no rows returned