Our customers made these suggestions for improving Etlworks Integrator. Suggestions for improving Etlworks Integrator.
44
votes
Packages and versioning
It would be nice to create a "package" of flows and connections and store it as a certain version. It should then be easy to switch between versions just by toggling ...
11
votes
GUI Database Lookup
As a user, creating database lookups requires the user to know code in the field value function to perform database lookups. This is even more complicated when doing ...
10
votes
SSO Authentication
Etlworks handles and processes a lot of sensitive data. It is important that this is adequately shielded and becomes part of our company-wide security umbrella. The ...
9
votes
Test Script button
Testing the „Execute JavaScript or Python code“ type flows is very troublesome, as you have to save (which closes the modal), press run, open the flow execution view, ...
8
votes
(re-)Run from error screen
If investigate an issue within ETLworks. You go to the error (via status) to the statistics section. When solving the issue, you have to search for the flow in order ...
8
votes
New fields' mapping editor
Enhance field mapping editor:
- add support for nested fields in the destination
- add an ability to modify attributes, such as JSON array, XML CDATA, XML ...
7
votes
Add retry functionality to agent schedules
ETLWorks host based schedules support: retries (1 - 5), delay between retries (1 - 10 minutes) and send failure email only if last retry failed. It would be really ...
7
votes
Provisioning from AWS and Azure Marketplaces
Add ability to provision Etlworks from AWS and Azure Marketplaces with Amazon and Microsoft billing
6
votes
Better ways to determine how flows, connections, etc. are interconnected
Right now when you have multiple flows that share connections or other flows, it's difficult to understand how making a change to one will possibly impact other ...
6
votes
IBM i Debezium Connector
Hello, would it be possible to add IBM i/AS400 Debezium Connector capabilities via this repo please?
https://github.com/jhc-systems/debezium-connector-ibmi
6
votes
Ability to schedule and execute flows by data integration agents running behind the firewall
Basically, add support for hybrid data integration when flows can be executed on data integration agents running behind the firewall.
6
votes
Ability to update all or selected agents to the specific version
Currently each agents needs to be updated individually.
From our perspective, for bulk updating, I would say it would be managed via the Agents page itself.
- ...
6
votes
Improved Filter/Search in Flow Statistics Page
It can be cumbersome to locate a specific job in the Flow Statistics page since it loads the jobs in the order in which it was run. Being able to search or filter ...
6
votes
Recycle Bin when deleting flows (or other assets)
It would be nice if there was a Recycle Bin for flows (or other assets), and it could have a time limit (like 30 days). This way, if one of our users accidentally ...
6
votes
Drag&drop SQL query builder and Field Function Bilder
- SQL Builder (in Flow and Explorer)
- Field Function Builder (at least Lookup)
5
votes
Implement priority queue and throttling for user-defined APIs (listeners)
Scenario 1: upstream system slammed the system with thousands of API calls in a short span of time, eventually freezing and requiring a reboot to stop all jobs ...
5
votes
Add exceptions/exclusions rules to schedules
Often times we need to apply schedules to not run at certain times for various reasons. Right now, we have to create multiple schedules. It would be nice to ...
5
votes
Search functionality / better documentation of flows
We use a good deal of javascript and in complex flows it can be really difficult to debug, esp. global and flow variables because there's no way to search (that I ...
5
votes
Truly dynamic connection and format
Currently adding dynamic connection and format requires actually adding a named connection to the flow. It would be great if both could be truly dynamic, resolved at ...
5
votes
Append Mode Option in Flows
Feature Name: Append Mode
Purpose: To provide a seamless way to append data to the destination without overwriting existing records.
Implementation Suggestion: A ...
5
votes
Statistics Executions: Sort and Filter Updates
On the statistics page, specifically the Executions area. The ability to:
1. Sort the results by the column headers. At least the started, ended, duration, and ...
4
votes
Ability to purge files within Server Storage connection after specified number of days
I would like to be able to delete files within a Server Storage connection after a specific number of days. This can be useful if I generate file logs into a specific ...
4
votes
Interactive Performance recommandations
It would be a good idea to get performance recommandations based on one or more (nested) flows. Goal is decrease (processing time) of the flows. Maybe this can be ...
4
votes
Connector for Google Analytics 4,
Since Google is discontinuing Universal Analytics and moving to only supporting Google Analytics 4,
4
votes
AWS Kinesis Integration
AWS Kinesis Integration
4
votes
Add message and script to scheduler email
I want to add a message to the scheduler that contains all the handled flow exceptions.
Add a message field where we can append additional text to the default email ...
4
votes
New mapping options
1. If configured throw an error if there is column in the source which does not exist in mapping
2. If configured ignore all columns in the source which are not in ...
4
votes
Filtering audit logs by time
Currently, audit logs can be filtered by the usable fields and a date range. Having a large amount of entry's, makes the log hard to use with these specifications, if ...
4
votes
Excluding schedules or flows from the execution notification
I do like having the execution notification in the top right corner of the page, however there are some scheduled tasks which run so frequently that it would make the ...
4
votes
View of schedules across all tenants + Last execution duraton
A view were all schedules across all tenants are visible with additional column with last execution duration will help balance load across instance. It will help pick ...
3
votes
Scheduler queue
Introduce queuing in scheduler so you can set how many scheduled tasks can execute simultaneously, and if one tries to run while another is running it will wait till ...
3
votes
Bulk insert with db to db flows
Having inserts into postgres and sqlserver work like redshift would be a nice improvement. Right now it runs a bunch of insert statements. Instead, pull source data ...
3
votes
Import entire schemes without specifying individual table
I really liked how aws dms operated, you pointed it at a schema on your source, told it the target schema in your destination, and had a wildcard supporting ...
3
votes
Table view in Test Transformation
1) If the destination connection is a database display output in a table view
2) Use code editor for a preview
3
votes
Ability to create and share templates
Templates are similar to export/import. Additionally, users should be able to:
- Create a template from any Flow/Connection/Format.
- Share templates with everyone ...
3
votes
Multi select flows for bulk actions (delete, add tags, move to tenant) etc.
Would like to be able to select via checkbox or "all" results of a search and perform a bulk action on a set of flows, connections, users, etc.
3
votes
Default tenant at login
Allow user to set default tenant (e.g., dev versus prod) when logging in. Currently defaults to no tenant.
2
votes
Statistics Page Should include Agent Metrics too
Currently Statistics page does not include metrics for the flows which are executed by Agent... it would make dashboard much better if we can see the whole picture ...
2
votes
Allow for table list alphabetization in explorer
Expanded salesforce tables in explorer, table names are not alphabetized, makes it hard to find anything
2
votes
Searching flows should also search table names
I want to find a flow that syncs a specific table, the search only includes the flow names. If it's easy include the table names or other sub-fields or scripts
2
votes
Add runtime per step (addition to start / stop)
A this moment we have to interpretate the flowstep running time by calculating Start / Stop . It should be handy if you could see this at once. We use this qiut often.
2
votes
Add ability to generate new Google Sheets
I'm looking for a way to generate new Google Sheets from ETLworks.
Or at minimum, use the Google Drive connector to create a new file, then the Google Sheets ...
2
votes
Can register with same email on multiple tenants
As administrator, I would like to use the same email on multiple tenants as opposed to having to create a new one per tenant. This would also require that email ...
2
votes
Implement Kafka Headers
Please consider implementing Kafka Headers. Thanks!