Network "Batch" Processing
Current large volume Batch Processing is often limited to one machine, they are run sequentially, have a single point of failure and provide no dashboard to monitor and scale on-demand. Process logging is non-existent or clunky.
Opturo’s Network Processing product provides a robust platform with a decentralized architecture, an Administration Console and one or many Operational Clusters. Following are the components:
• An Operational Cluster consists of one “Manager/Broker” and multiple “Worker” daemon processes. Worker daemon within a Cluster can run on one or more machines on the network.
• A Manager/Broker within a Cluster maintains a list of “Queues” and distributes “Jobs” to Workers.
• Each Worker/Consumer daemon subscribes to Jobs on one or more Queues and processes them.
• A Batch Process with one or more “Iterations” or “Jobs”, is initiated by a “Producer” which sends a message to the Manager with list of Jobs/Iterations and a Queue name.
• A Job/Message or Iteration is one loop of any process, such as calculating analytics on 20 accounts (20 Iterations) or processing 100 files (100 Iterations).
Using this architecture, the processing of any Batch Process with one or more Jobs/Iterations can be distributed across multiple machines on a network. This drastically cut run times and improves reliability.
Administration Console allows real-time monitoring current Batch Processes. Provides the ability to scale Workers on-demand to handle larger volumes.
The Opturo Advantage
Integrated with VIA
New custom Batch Processes are created quickly using Opturo’s VIA IDE Application which leverages the robust data management/custom solution/third-party integration functionality of the VIA Platform.
No limitation on scalability
There is no known upper limit on scaling a Batch Process using this product. The product can be scaled to meet the demand of a 1K or 1B Iteration Process. The only constraining factor is hardware.
High Availability and Redundancy
No single point of failure. Workers can be designated as back-up Manager/Broker which is activated if the main Manager/Broker is offline.
Web-based Dashboard Manager
Real-time monitoring through web-based Administration Console of Batch Processes and machines on the Cluster, allows users to effectively manage the process. System allows scaling on-demand by having the ability to start new Workers on a remote machine.
Priority Queues
Use of Priority Queues allow for time-sensitive Batch Processes to take precedence.
Detailed Real-time and Historical Logs
Provide access to current and Historical Iteration-level logs with real-time machine health statistics.
Key Benefits
Implementation
Opturo’s various analytical solutions can be implemented in a fraction of the time of competing systems, as we leverage each client’s unique infrastructure. Data formatting, replication and synchronization issues can be minimized or eliminated with an Opturo implementation.
Multi-Language
Users may select the language they prefer when utilizing the platform, which includes the web-based GUI and reports exported into PDF files. Don’t see your preferred language in those we offer? We are able to rapidly add client requested languages that are not currently supported.
Speed and Scalability
Opturo’s innovative architecture provides clients with a platform which can be scaled to meet the needs and scope of any project. Whether leveraging significant proprietary data infrastructure or going to the cloud, Opturo can help the largest enterprises address their big data challenges.
Advanced Reporting
Opturo offers clients the industry’s most robust suite of reporting and visualization tools. Transparent data and model calculations allow cascading time series drill-down visualization. Clients are provided the tools to create custom reports themselves, or they can leverage our professional staff to assist.
Flexible Customization
Is your infrastructure robust enough to meet the challenges and opportunities the future may present? Opturo offers our clients a modular platform which is inherently flexible and able to respond to emergent trends and requirements, whether it be for data management, reporting or analytical needs.
Cloud or Local Deployment
Opturo can be deployed in the cloud or within a client’s existing infrastructure. Opturo is agnostic and works with each client to insure a deployment that makes the most sense for each client’s business priorities, whether they are regulatory-driven or business model preference.
