Suggestions for improving Etlworks Integrator
45
votes
Packages and versioning
It would be nice to create a "package" of flows and connections and store it as a certain version. It should then be easy to switch between versions just by toggling ...
12
votes
GUI Database Lookup
As a user, creating database lookups requires the user to know code in the field value function to perform database lookups. This is even more complicated when doing ...
12
votes
Drag and drop process builder for nested flows
Replace existing nested flow editor with drag and drop process builder:
- works similarly to the diagram editor
- ability to select existing flows from the side-bar ...
10
votes
Ability to update all or selected agents to the specific version
Currently each agents needs to be updated individually.
From our perspective, for bulk updating, I would say it would be managed via the Agents page itself.
- ...
10
votes
SSO Authentication
Etlworks handles and processes a lot of sensitive data. It is important that this is adequately shielded and becomes part of our company-wide security umbrella. The ...
9
votes
Test Script button
Testing the „Execute JavaScript or Python code“ type flows is very troublesome, as you have to save (which closes the modal), press run, open the flow execution view, ...
9
votes
Drag&drop SQL query builder and Field Function Bilder
- SQL Builder (in Flow and Explorer)
- Field Function Builder (at least Lookup)
8
votes
Recycle Bin when deleting flows (or other assets)
It would be nice if there was a Recycle Bin for flows (or other assets), and it could have a time limit (like 30 days). This way, if one of our users accidentally ...
8
votes
(re-)Run from error screen
If investigate an issue within ETLworks. You go to the error (via status) to the statistics section. When solving the issue, you have to search for the flow in order ...
8
votes
New fields' mapping editor
Enhance field mapping editor:
- add support for nested fields in the destination
- add an ability to modify attributes, such as JSON array, XML CDATA, XML ...
7
votes
Access Control for Shared Components in ETLWorks
ETLWorks would have the capability to lock a selection of formats, flows, and connections to particular user groups. This is because they can be repeatable shared ...
7
votes
Add retry functionality to agent schedules
ETLWorks host based schedules support: retries (1 - 5), delay between retries (1 - 10 minutes) and send failure email only if last retry failed. It would be really ...
7
votes
Excluding schedules or flows from the execution notification
I do like having the execution notification in the top right corner of the page, however there are some scheduled tasks which run so frequently that it would make the ...
7
votes
Provisioning from AWS and Azure Marketplaces
Add ability to provision Etlworks from AWS and Azure Marketplaces with Amazon and Microsoft billing
6
votes
Better ways to determine how flows, connections, etc. are interconnected
Right now when you have multiple flows that share connections or other flows, it's difficult to understand how making a change to one will possibly impact other ...
6
votes
Saved Views Based on Tags
We’d like to request “Saved Views” on ETLWorks pages that support tags- starting with Schedules, Flows, Connections, Agents, and Explorer pages.
- These views ...
6
votes
Filtering audit logs by time
Currently, audit logs can be filtered by the usable fields and a date range. Having a large amount of entry's, makes the log hard to use with these specifications, if ...
6
votes
Improved Filter/Search in Flow Statistics Page
It can be cumbersome to locate a specific job in the Flow Statistics page since it loads the jobs in the order in which it was run. Being able to search or filter ...
6
votes
IBM i Debezium Connector
Hello, would it be possible to add IBM i/AS400 Debezium Connector capabilities via this repo please?
https://github.com/jhc-systems/debezium-connector-ibmi
6
votes
Ability to schedule and execute flows by data integration agents running behind the firewall
Basically, add support for hybrid data integration when flows can be executed on data integration agents running behind the firewall.
5
votes
Implement priority queue and throttling for user-defined APIs (listeners)
Scenario 1: upstream system slammed the system with thousands of API calls in a short span of time, eventually freezing and requiring a reboot to stop all jobs ...
5
votes
Add message and script to scheduler email
I want to add a message to the scheduler that contains all the handled flow exceptions.
Add a message field where we can append additional text to the default email ...
5
votes
Append Mode Option in Flows
Feature Name: Append Mode
Purpose: To provide a seamless way to append data to the destination without overwriting existing records.
Implementation Suggestion: A ...
5
votes
Statistics Executions: Sort and Filter Updates
On the statistics page, specifically the Executions area. The ability to:
1. Sort the results by the column headers. At least the started, ended, duration, and ...
5
votes
Interactive Performance recommandations
It would be a good idea to get performance recommandations based on one or more (nested) flows. Goal is decrease (processing time) of the flows. Maybe this can be ...
5
votes
Default tenant at login
Allow user to set default tenant (e.g., dev versus prod) when logging in. Currently defaults to no tenant.
5
votes
Repeat long running schedule notifications
When telling a schedule to alert you if it’s been running for more than X minutes, allow it to alert you every X minutes. Right now I can’t tell the difference ...
5
votes
Add exceptions/exclusions rules to schedules
Often times we need to apply schedules to not run at certain times for various reasons. Right now, we have to create multiple schedules. It would be nice to ...
5
votes
Search functionality / better documentation of flows
We use a good deal of javascript and in complex flows it can be really difficult to debug, esp. global and flow variables because there's no way to search (that I ...
5
votes
Truly dynamic connection and format
Currently adding dynamic connection and format requires actually adding a named connection to the flow. It would be great if both could be truly dynamic, resolved at ...
4
votes
Ability to purge files within Server Storage connection after specified number of days
I would like to be able to delete files within a Server Storage connection after a specific number of days. This can be useful if I generate file logs into a specific ...
4
votes
Direct URL to Specific Flow Execution within History
Currently, when investigating a specific execution in ETLWorks, users must manually locate it within the execution history page. This process is time-consuming, ...
4
votes
Multi select flows for bulk actions (delete, add tags, move to tenant) etc.
Would like to be able to select via checkbox or "all" results of a search and perform a bulk action on a set of flows, connections, users, etc.
4
votes
Connector for Google Analytics 4,
Since Google is discontinuing Universal Analytics and moving to only supporting Google Analytics 4,
4
votes
FLOWs visual based on tree hierarchy
When having multiple flows, it would be nice if you can collapse actions into the nested flows (like a tree).
Result then, only nested flows will be shown, until ...
4
votes
AWS Kinesis Integration
AWS Kinesis Integration
4
votes
Can register with same email on multiple tenants
As administrator, I would like to use the same email on multiple tenants as opposed to having to create a new one per tenant. This would also require that email ...
4
votes
New mapping options
1. If configured throw an error if there is column in the source which does not exist in mapping
2. If configured ignore all columns in the source which are not in ...
4
votes
View of schedules across all tenants + Last execution duraton
A view were all schedules across all tenants are visible with additional column with last execution duration will help balance load across instance. It will help pick ...
3
votes
Scheduler queue
Introduce queuing in scheduler so you can set how many scheduled tasks can execute simultaneously, and if one tries to run while another is running it will wait till ...
3
votes
Super Admin–Managed Environment Variables
Requests:
Phase 1: Add environment-scoped variables editable only by Super Admins; referenceable via (like `{VAR_NAME}`) in flows, connections, etc
Use cases: ...
3
votes
Interactive Statistics
Possibility to click on a column in statistics and get the corresponding values in the rest of the statistic (Instead of choosing the date from the Datelist).
If you ...
3
votes
Order of flows in agent\schedule
It will be great if there be ability to order the flows in agent\schedule panel
3
votes
Bulk insert with db to db flows
Having inserts into postgres and sqlserver work like redshift would be a nice improvement. Right now it runs a bunch of insert statements. Instead, pull source data ...
3
votes
Import entire schemes without specifying individual table
I really liked how aws dms operated, you pointed it at a schema on your source, told it the target schema in your destination, and had a wildcard supporting ...
3
votes
Table view in Test Transformation
1) If the destination connection is a database display output in a table view
2) Use code editor for a preview