Migration from trial notebooks
- How do I migrate my trial notebook to the new MLHub?
All information on the migration can be found in this document. Here's a quick summary of the steps:
1. Export the notebooks that need to be kept before starting the upgrade process
2. Convert the notebooks, using a script that your Customer Success Manager can provide
3. After the upgrade, import the converted notebooks. This needs to be done by the user who will own the notebook.
Make sure to take the necessary actions before your TrendMiner installation is upgraded to version 2022.R2.0. For more information, contact your Customer Success Manager or support@trendminer.com. - Will my Machine Learning Model tags still work after the upgrade?
Machine Learning Model tags are automatically migrated during the upgrade to TrendMiner 2022.R2.0 if there is an active license for notebooks and Machine Learning Model tags. Make sure to check this before starting the upgrade process (more information on how to check can be found here. - My DashHub trial notebook tiles don’t work anymore, what do I do?
DashHub tiles based on trial notebooks are not automatically migrated and need to be recreated. Delete the tiles from your dashboard and follow the procedure in this article to create a pipeline object from your (migrated) notebook and configure a new notebook output tile using this pipeline. Note that this step can only be completed after the upgrade and migration of the notebooks have been completed.
MLHub notebooks
- What scripting languages are supported in the notebooks?
Currently, only Python is supported. Other languages are planned to be added in future releases. Use the ‘Make a wish’ function in the Help menu to let us know what languages you would like to use in notebooks. - How can I install other languages?
It is not possible to install other languages yourself. Other languages are planned to be added in future releases. Use the ‘Make a wish’ function in the Help menu to let us know what languages you would like to use in notebooks. - Which packages can I use?
The following Python packages are shipped with MLHub by default: dask, ipykernel, ipython, Jinja2, Keras, matplotlib, nbformat, numba, numpy, nyoka, pandas, pandasql, plotly, protobuf, psutil, pyod, scipy, seaborn, scikit-learn, statsmodels, suod. If your TrendMiner server has an open internet connection, you can also install additional packages. - How can I install more packages?
You can install additional packages by using the pip install command in your notebook code. Note that the TrendMiner server needs to be connected to the internet for this to work (please check with your local administrator if this is the case). Using this command, the package will be downloaded in your current kernel. Example: “pip install tensflow” will download and install the latest TensorFlow package. It is not possible to install additional packages if your TrendMiner server is not able to connect to the internet. - Can I import existing Python scripts?
You can import notebooks using the import button in MLHub. Only Jupyter notebooks (file extension .ipynb) are supported. - Can I export my notebooks?
The notebook toolbar has an option to download your notebook (file extension .ipynb). - How can I collaborate with a colleague on a notebook?
You can share your notebook with a colleague via the Work Organizer. Note that your colleague will only have view rights to the notebook and changes they make will not be saved. Sharing with edit rights is planned to be added in the future. Use the ‘Make a wish’ function in the Help menu to let us know if this would be useful to you. - What data does MLHub use when loading a TrendHub View?
MLHub will use indexed data, as saved by TrendMiner. The code snippet will grab the data stored in the view and import it as a dataframe in your notebook. - How do I know my notebook was successfully saved?
You need to manually save your notebook after any changes made. Confirmation messages are planned to be added in future releases. - Can I persist the TrendMiner content snippets across working sessions?
Currently, these snippets are only available in your current session and not persisted. This means that your TrendMiner content menu will be empty after reloading the page. - How can I delete a cell in my notebook?
There is currently no pure ‘delete cell’ option. You can however use the ‘cut’ action to remove the cell. Note that this overwrites your current clipboard content. - Can I run my whole notebook without restarting the kernel?
No, you can either run the notebook cell by cell or choose the option to restart the kernel and run the whole notebook (this is only one click though). - Why are my notebook dialog boxes in English instead of my browser language?
TrendMiner uses third party technology for the MLHub notebooks. Even though most capabilities are locale sensitive we cannot guarantee every dialog box and text will be fully translated for these notebook capabilities.
DashHub notebook tiles
- How do I share notebook output on a dashboard?
You can share notebook output on a dashboard in DashHub via a notebook output tile. Create a pipeline object using the ‘Publish’ option on the MLHub notebooks (more information here). Next, add a notebook output tile to your dashboard using the created pipeline. More information on setting up DashHub tiles can be found here. - How do I edit the pipeline used in my DashHub notebook tile?
Currently it’s not possible to edit or overwrite existing pipelines. You need to create a new pipeline from your notebook and assign it to the notebook output tile.
General best practices
- Do you have any general best practices to ensure the best performance of our notebooks?
The following tips should help you ensure your notebooks run performantly:- Do not unnecessarily load large datasets. Only include the necessary tags and time periods in the views you will load into your notebooks.
- g., if you want to show current data compared to a calculated regression line, do not include the regression line calculation steps that loaded potentially months of data. Instead, save the regression line after parameter tuning and draw it immediately.
- When creating a pipeline object to use in a DashHub notebook tile, only include the necessary steps of your notebooks. Omit all steps that do not contribute to the final output you want to share.
- Separate model training from model execution. Do not re-train a model on every execution of a DashHub notebook tile. Instead, just execute the model on the new data available.
- Do not unnecessarily load large datasets. Only include the necessary tags and time periods in the views you will load into your notebooks.
Performance
- What are the recommended system resources?
The recommended system resources for TrendMiner incl. MLHub can be found in the installation guide. Note that system resources may have to be increased further when there is heavy use of the MLHub capabilities. - How much memory is allocated to my notebook and my DashHub notebook output tile?
Every notebook kernel will get 1GB of RAM allocated. The kernel will be purged after 30 minutes on non-use. Notebook output tiles get 256MB of RAM allocated when they get refreshed, which will be purged directly after execution of the connected pipeline. Please refer to the general best practices above to avoid running into memory constraints. - How many users can use notebooks concurrently?
This will depend on the available system resources and the number of notebook tiles being executed. On the minimal system requirements (see installation guide, there should be no issue to have at least 4 concurrent users use the notebooks. In case you notice performance issues when more users are using MLHub or with large amounts of notebook output tiles created, the system resources (RAM and CPUs) will have to be increased. - What is the impact of MLHub on the core TrendMiner performance?
MLHub, notebooks and notebook output tiles use their dedicated resources, independently from the core TrendMiner services. Hence under normal circumstances, MLHub will not impact the core TrendMiner performance. In exceptional cases though, a failure of another TrendMiner service while MLHub is requesting a lot of memory, may result in a restart failure of the other service.
Troubleshooting
- Why can’t I open a notebook from my Work Organizer?
Opening a notebook from the Work Organizer is currently not supported. We are planning to add this in a future version. You can open your notebook by using the ‘Load’ option in MLHub. - What should I do when I get syntax or other python specific errors?
If the error occurs on a loaded TrendMiner snippet, feel free to reach out to support@trendminer.com to investigate the problem. Syntax or other python specific errors on your own code are your responsibility to debug. - Why did my code disappear after closing the notebook?
MLHub notebooks do not auto-save. You need to explicitly save your notebook before closing it. We recommend saving your changes regularly. - Why do I get a “File save error” in the notebook?
This typically happens when you have deleted the notebook from the Work Organizer while it was still open in the editor. - What should I do when I get the error “Something went wrong” in a notebook output tile?
Ask the owner of the notebook to check if all necessary cells are included in the pipeline object. If the code is loading a large view or contains complicated code, it could be that the memory assigned to the tile is not sufficient. Alternatively, there might not be enough resources available to run all current notebooks and tiles simultaneously.
Please reach out to support@trendminer.com after performing some initial checks on pipeline completeness, code complexity and number of notebook users / notebook tiles on this installation. This information will help us pinpoint the cause and suggest a solution more easily.