Wednesday, October 20, 2010

SmartGWT vs. Ext GWT ( Or 80-20 rule of development )



When you choose the framework you plan to work with, you try to select the best framework. It means that the framework has all required features, is flexible for extensions, has clear code, has rich documentation, is easy to use, has consistent APIs, has strong usage community , has  good support ,easy deployment, performance , memory issues is handled, and for sure ,has potential for future grow and new features implementation.
In the Web UI world, such framework does not exist and probably such framework doesn’t exist at all.   We need to understand that our expectations will never be fulfilled for 100% .We need to choose the framework at the begging of the project and at this point we still don’t know which extreme UI features our Functional Architect will require. We can just estimate, but from my experience the real final thing that really works and can answer on customer requirement may be far away from the initial estimation.  Anyone who has experienced with UI development will agree with me, since UI has the most dynamic changes in between versions, weekly and daily. So ,we just can’t expect from the selected framework to be perfect, and I am Ok with this as long as this framework answers on 80% of my expectations and allows me to extend it as I wish. These 80% include rich set of features, clear code and APIs, flexibility for extra extension, rich documentation, easy use and performance and memory issues handling. The limitations that I may allow are: The framework doesn’t need to be at the top of the GWT technologies development, but however it should be clear from the community forums that it goes there and in future versions all new features will be handled. In other words, it should be on the Technology Front, but it is not need to have all implementations right now. As well, I don’t expect it to have all UI features that I can  think about, but rather I expect it not to limit any feature development, allow components extensions and access to system native APIs.
In my research and conclusions regarding existing GWT UI frameworks I followed the concepts described above. My experience with SmartGWT is longer than with Ext GWT, I started with SmartGWT about a year ago and developed several screens for UI MashUp system, and recently I created a whole mockup with complete application layout, skinned and rich of ‘Wow’ effects of animation and fadeout-fadein. After I drilled down to the head of this library I turned to find something better. Recently I created the similar system with Ext GWT in order to compare my impressions. My purpose was to evaluate the feasibility of all of the features I implemented in the first project and also to test Ext GWT API in order to learn it better.
When I started, first of all I tried to find the most popular frameworks by the lists given by Wikipedia and other public sources. I rejected almost all frameworks because they were not rich in enough features, so I only selected two libraries, for checking: SmartGWT, ExtGWT, both work with GWT 2.0.

General APIs impression
o   The impression is that SmartGWT has exaggerated usage for their  GWT Widget wrapper object (the Canvas object):
       All APIs receive Canvas, that’s why, it actually makes it impossible to use native GWT components inside SmartGWT components. So the main impression is that SmartGWT is completely separated library ,which includes a really rich set of features but also obliges you to use their only components. For me it’s a limitation, because again my assumption is that any library ,just can’t include all  features in the world ,and in some stage in my development it will be necessary to use more components then only SmartGWT components. Also it makes my code depends forever on SmartGWT. I know that other libraries will make my code to be depend as well on there libraries, but if they supports GWT native objects it will be easily to decouple the code from them.
o   The example for such APIs: addMember(Canvas) of HStack object or method setPane(Canvas) of Tab.
o   In addition to this dependency, the APIs are not consistent; adding new member to the container element differs from object to object, as in the above example with layout object and tab. Both methods mean the same functionality – adding a new child to the container, but for some reason have different names: once it’s called “addMember” and another called “setTab”. 

These are just small things, but when you deal with big project and with thousands lines of code, it became very annoying.
There were no such problems with Ext GWT, the APIs are very close to GWT APIs, all receive native object and are consistent.

Effects
I wanted to implement something similar to IPhone panel, when you see one screen the second flies over it from the side. In both libraries I didn’t find a good way to do it. In both of them, I discovered problems when running animation on object which is inside the container. I still continue implementing it with Ext GWT, so now, it seems that I will need to create some floating absolute positioned panel and just move it over the first screen; this is exactly what I did with SmartGWT.
The other effects like fade-in-fadeout, rich set of move and other animation effects, exist in both libraries.

Layout APIs
Ok, the Ext GWT has very good Layout set, I tested almost all layouts and it works fine. From the begging it seemed to me too much complicated, too many different layouts: FitLayout , BorderLayout, CardLayout, CenterLayout etc, but after several tests I liked the structure and the concepts. It just worked as expected.
In SmartGWT there are two main layouts: HStack and VStack, from the first view it seems that everything is possible to create with these two basic stones, but somehow it has a lot of problems. The problems that I saw mostly were related to screen stretch layouts and alignment problems. Till now I didn’t succeed to align image to the center of the Canvas, if you know how to do it I will be happy to get the code sample. Also the Canvas basic object for some reason has 100px height by default, so if you want to create a smaller object, you must set the exact pixels, this is not good enough and to solve it I did some ugly workarounds. 

Events model
Ext GWT 2.0 still uses Listeners and it require some work to implement the MVP pattern. These Listeners are something between old GWT listeners and new Handlers pattern and in their roadmap they promise to implement new GWT Handlers mechanism in the coming 3.0 version.
SmartGWT uses GWT Handlers approach.

UIBuilder
UIBuilder is not supported. As I understood from Ext GWT forums this is not because of the library, but because of UIBuilder is not yet opened its internal API for overriding. However because of the fact that Ext GWT APIs are closer to GWT native APIs, I believe that once it will be possible it will be supported quickly by Ext GWT, while for SmartGWT is far away because of the code wrappers and other APIs limitations described above. 

Look&Feel Flexibility
Both libraries work with well defined skin. In SmartGWT the skin has code JavaScript extention, it consist of style css file and javascript file there you can add more logic, for example you can manage more UI controls modes than exist in css like toggled buttons. The Ext GWT skin contain just one style file, as simple as it should be for my opinion. 

Component set
Very rich in both libraries, SmartGWT has more built-in functional components, but Ext has more rich Layouts.

Performance issues
This is the most painful point for SmartGWT, because SmartGWT is not built as GWT native code, but is simply a wrapper above existing JS library. That’s why, it can’t use GWT compiler performance optimizations (like code split points). Yep, despite the fact that somebody already tried to convince me that the performance in general is OK and already have tested on big systems I think this is really big minus. GWT is quick growing technology and from version to version Google add more features, so choosing SmartGWT will mean that no core compiler features may be used in the future.  

Support and Community
When I downloaded Ext GWT I started asking the most stupid questions on their forum, about how to run the library and its components. In all cases I received immediate quick answers. For me this is a good sign. Also Ext GWT has a support package that includes online and offline support and even private patches (this service cost money). So, with whatever you need, you can get an urgent help, just payJ )
SmartGWT has also a big forum; it takes up to several days to answer, not bad for open source library.

License
SmartGWT is open source. Ext GWT has commercial different packages license from general to platinum, the license is per developer without royalty and also open source version that can be used for open sources products

My Conclusion
Ok, so where are the drawbacks of Ext GWT? My answer is that they are just there; I didn’t have the chance to discover a lot of them. Because I’ve started with SmartGWT and experienced all drawbacks of it, I only tested the same features for the second library to insure that it is built in the different way. So probably some problems will arise and I don’t expect it to be perfect, but again on the rule of 80-20 the Ext GWT beats SmartGWT in almost all aspects. 

Alona Oz
Web UI Expert

Thursday, October 14, 2010

Tips for developing the best High frequency trading (HFT) system.

1) When developing your HFT system , make sure you have an order execution engine that mirrors how the market acts , which will help you test/debug your platform/strategy in your own computer. The 'test mode' that usually supplied by the exchanges is not a replacement for that. Remember that your main interest is to preform all unit testing in your own environment , before you even start testing on the server side. Any Developer knows that its a big headache to debug on remote server ,especially when the far server do not have a debugging environment. For eg: Think about a situation when you are testing remotely in the server and you get a crash ,which caused because your system cannot deal with the low latency - Now ,your immediate response will be to put some prints and find where is the bug that cause the crash - but if you put some prints you suspend the system ,and the bug will not be there anymore - Now , how can you solve this problem remotely ?

2) Latency is the most important issue in HFT , That's why your code needs to be tight, fast and rock solid stable. Having an Algo crash and being caught with shares when the market moves against you is not very profitable . Well-written code runs fast, whereas poorly written code runs slow. Furthermore, well-written, object-oriented code will be far easier to debug and maintain.

3) The best Latency can be achieved by using hardware-accelerated functions - In other words , instead of using high level language - your platform+strategies will be converted to custom block of hardware that can run many times faster than the same code optimized in tight assembly language. But ( There is always But ) , you can only get there after you wrote the best optimized code by yourself , and then you will get some ideas about the parts you want to optimize.

4) Before even start coding , think about all the bottlenecks that will come letter . Find the solutions before they appear .Design your system in a way that it can be expended easily without damaging the latency.As you know sooner or later you will have to add more features - make the design ready for any future requirements.

5) Use the power of the hardware you have - make sure your software can take the advantages of the hardware you have.Design your software in a way that if tomorrow you get a new powerful hardware , It will boost the performance of your software's performance without the need of changing your software, because your software was designed from the beginning , to naturally fit your hardware architectures.

6) Be sure to be armed with the latest technology - That's what can make the difference between micro and nano sec. Change your development approach and start thinking of using parallel programming standards such as MPI for processors communication or other advanced techniques for Shared Memory messaging (This days , you can get to 700 nanoseconds of Latency for IPC). By using Advanced process techniques ,you can identify patterns among streaming event data and respond to those patterns in microseconds , This makes it possible to monitor, analyze and act on market data and respond immediately. You can use neural network for stock market prediction , Evolutionary Computation techniques, such as genetic programming to analysis of financial markets with a reassuring degree of success ,and fuzzy logic to find patterns and trends in financial indices. By Using advance A.I. technologis , the experience of a trader can be incorporated in a natural way inside your system.

Moris oz
Algorithmic trading Consulting and development
www.caapitech.com

Tuesday, October 5, 2010

The Benefits of outsourcing 'Algorithmic Trading' projects

(1) Independent Traders - Some traders wants to test there strategies using HFT structure, after testing with basic tools. They need a flexible HFT system with a mechanism for easily integration of various strategies within the system, with a full ability to add features that they need, with a "personal developer" that give them the right support they need.

(2) Setup A business - If you ever run an HFT System , you know that first you have to set up the legal and administrative structure and then set up the physical trading infrastructure ( hardware , server hosting, collocation servers - Especially in HFT you want to locate your servers near an Internet backbone as close as possible to the exchange on which your trades will be executed) , and you already spent around 100,000$ in the good case , without even start developing any system yet.Recruiting a development team will cost you extra million $ and you still have no profit yet.

 I recommend to my clients:
(1) Establish your business - First Establish your business :Get a freelance talent translates into cutting-edge skills and fresh perspectives to help get jobs done faster and better.Freelancers can help complete projects quickly, without the cost of additional overhead that full-time employees require. By finding the right professional programmer that already build an HFT infrastructure before ,you start a project right away and Give your business a competitive edge , instead of wasting big amount of money and weeks or months to hire people, train them, and provide the support they need. And if talking about HFT system that requires major capital investments , the start-up process can be even more difficult.
(2) Building one step at a time - I believe in Building one step at a time ,and go on to the next step after the previous one have been tested successfully.To start small, with limited investment and gradually scale up the business as you gain know-how and become profitable.It will be much easier to get more capital invested in your system after you have A simple HFT system with basic strategy that is working well and making some money. After you get more capital , you can invest in finding good physicists, mathematicians to get you a better strategies and if needed find some developers to add/optimize more features in your system.
(3) Focus in the core business - As mentioned before , In the 'Algo trading' business there is much more then development .I believe that my clients should Focus in there core business and let me do the development and to find the technical solution for them .As you know Every business has limited resources, and every manager has limited time and attention. A Good freelancer can help managers set their priorities more clearly.
(4) Risk management -  Every business investment carries a certain amount of risk. Outsourcing providers assume and manage this risk for you, and they generally are much better at deciding how to avoid risk in their areas of expertise

Moris oz
Algorithmic trading Consulting and development
www.caapitech.com

Monday, October 4, 2010

How to choose the best Algorithmic Trading programmer ?

1) Communication : When choosing someone to do any type of programming work for you, make sure you can communicate with him or her. After All - you want them to understand what you want and need.

2) Experience : You need to figure out whether the person is really capable of doing the work. In other words, make sure that he aren't the kind that just "talks the talk".Building an Algorithmic Trading machine requires a very professional programming skills.. This is especially true for high-frequency strategies where the speed of execution is of the essence.
Make sure your Developer have an in-depth knowledge of trading technology , and he already did something like that before.

3) Efficient : Sometimes you have a basic idea of what you want and need ( For Eg: I want an HFT Algo Machine that can trade Forex ) , But you don't really know how to go about doing that. Exchange , DMA , Brokers , Protocols , Backtesting , hardware , Software , etc.. - let the programmer help you with that - let him determine what would really help you be more efficient.

4) Flexibility: Some customers know exactly what they want and can put it all in writing. This ability enables the customer to require that the programmer give an exact quote on the project. When the project is completed, the customer should receive precisely what was requested. If the customer is not quite firm on what is needed , then it will most likely be necessary to implement changes during the creation stage. These changes in direction require a flexible programmer. The programmer should be able to handle a reasonable amount of such changes without having to increase the bid. However, if a programmer lacks flexibility it will probably be impossible to get changes implemented without a new quote being made. Of course, this process involves additional cost and time.

5) Price : software is part art, part science, and part math. its a craft that requires more then 10 years to master.Would you hire the cheapest doctor, lawyer, or plumber you could find? If you take trading seriously and think of it as a business, this makes no sense.As they say you get what you pay for. hire a pro, he'll have it done in 1/4 the time and it will be right the first time. In Algorithmic Trading machine - Any Bug can be crucial , especially If talking about HFT systems - if your system is not fast enough , then you are out of the game.

Moris oz
Algorithmic trading Consulting and development
www.caapitech.com