Current large volume Batch Processing is often limited to one machine, they are run sequentially, have a single point of failure and provide no dashboard to monitor and scale on-demand. Process logging is non-existent or clunky.
Opturo’s Network Processing product provides a robust platform with a decentralized architecture, an Administration Console and one or many Operational Clusters. Following are the components:
• An Operational Cluster consists of one “Manager/Broker” and multiple “Worker” daemon processes. Worker daemon within a Cluster can run on one or more machines on the network.
• A Manager/Broker within a Cluster maintains a list of “Queues” and distributes “Jobs” to Workers.
• Each Worker/Consumer daemon subscribes to Jobs on one or more Queues and processes them.
• A Batch Process with one or more “Iterations” or “Jobs”, is initiated by a “Producer” which sends a message to the Manager with list of Jobs/Iterations and a Queue name.
• A Job/Message or Iteration is one loop of any process, such as calculating analytics on 20 accounts (20 Iterations) or processing 100 files (100 Iterations).
Using this architecture, the processing of any Batch Process with one or more Jobs/Iterations can be distributed across multiple machines on a network. This drastically cut run times and improves reliability.
Administration Console allows real-time monitoring current Batch Processes. Provides the ability to scale Workers on-demand to handle larger volumes.
New custom Batch Processes are created quickly using Opturo’s VIA IDE Application which leverages the robust data management/custom solution/third-party integration functionality of the VIA Platform.
There is no known upper limit on scaling a Batch Process using this product. The product can be scaled to meet the demand of a 1K or 1B Iteration Process. The only constraining factor is hardware.
No single point of failure. Workers can be designated as back-up Manager/Broker which is activated if the main Manager/Broker is offline.
Opturo’s various analytical solutions can be implemented in a fraction of the time of competing systems, as we leverage each client’s unique infrastructure. Data formatting, replication and synchronization issues can be minimized or eliminated with an Opturo implementation.
Users may select the language they prefer when utilizing the platform, which includes the web-based GUI and reports exported into PDF files. Don’t see your preferred language in those we offer? We are able to rapidly add client requested languages that are not currently supported.
Opturo’s innovative architecture provides clients with a platform which can be scaled to meet the needs and scope of any project. Whether leveraging significant proprietary data infrastructure or going to the cloud, Opturo can help the largest enterprises address their big data challenges.