Difference between revisions of "Web Application Development"

From MythTV Official Wiki
Jump to: navigation, search
(Some initial thoughts on developing the new WebApp)
 
(Update on Angular and PrimeNG versions)
 
(23 intermediate revisions by 3 users not shown)
Line 1: Line 1:
−
{{Incomplete}}
 
  
 
= Basic instructions for working on the new WebApp =
 
= Basic instructions for working on the new WebApp =
  
 +
== Getting started  ==
 +
The WebApp uses [https://angular.io/ Angular], [https://primeng.org/ PrimeNG], [https://primeflex.org/ PrimeFlex] and [http://www.ngx-translate.com/ ngx-translate].
 +
 +
Angular is the web framework we use.
 +
 +
PrimeNG is an Angular UI Component Library providing various form controls like text edits, check boxes, radio buttons, sliders as well as tables, panels, overlays etc.
 +
 +
To see the Version 16 documentation for PrimeNG, use https://primeng-v16.pages.dev/installation . The documentation is no longer on the main PrimeNG page.
 +
 +
PrimeFlex is a lightweight responsive CSS utility library to accompany Prime UI libraries.
 +
 +
ngx-translate handles loading translation.
 +
 +
== Development vs Production ==
 +
 +
We have intentionally checked in the production build artifacts. This is because installing the build requirements uses a method not permitted by the packaging process of Linus distributions. The "npm install" command is a quick and easy way to download and install into your directory all of the dependents. This cannot be run by a Linux distribution build because it accesses the internet.
 +
 +
=== Install Development Software ===
 +
 +
To check what version of Angular and PrimeNG the web app is using, look at mythtv/mythtv/html/backend/package.json. The Angular version to look for is Angular/cli. The major version of Angular/cli must match the major version of PrimeNG. The high level version number of all angular components should be the same.
 +
 +
To see what version of Node.js is needed, see the table at https://angular.dev/reference/versions. In that table and in package.json, version number ^x.y.z means any version x.*.* as long as it is at least x.y.z. ~x.y.z means any version x.y.* as long as it is at least x.y.z.
 +
 +
You can install nodejs and npm from your linux distribution package manager, or install it from the web. I prefer to install it from the web in order to have control of the version being used
 +
 +
Install node js into a private directory. This way you can run two versions at once if needed.
 +
 +
* https://nodejs.org/en/download/package-manager
 +
* Select the highest version that supports the version of @angular/cli from package.json
 +
* Follow instructions on that page
 +
 +
You now have node installed in $HOME/.nvm and entries in .bashrc that cause your version to run when you run node. You can type node at a command prompt and have an interactive javascript interpreter where you can test javascript commands.
 +
 +
You can install multiple versions of nodejs. Then use the command "nvm use nn" to switch the currently used version.
 +
 +
Install packages used by the web app:
 +
 +
* cd mythtv/mythtv/html/backend
 +
* npm install
 +
 +
This installs all required packages into subdirectory node_modules. It uses the package.json and other .json files to determine which packages to install. It will install all the required dependencies and ensure you are using the same versions as other developers. Occasionally you may need to re-run this if additional dependencies are added or updated
 +
 +
Do not install angular globally, in case you need to have different angular versions for different project versions.
 +
 +
Do not install node, npm or related packages from the system repository (apt get or similar). If you have, uninstall them.
 +
 +
=== Upgrading versions ===
 +
 +
We are currently using Angular version 16 and PrimeNG version 16. I have attempted to upgrade to version 17 and then to version 18. PrimeNG version 17 has minor changes but some things were not working when i attempted it, notably the lazy load feature used in several pages was no longer working. This feature seems to break with each new version. PrimeNG version 18 is a major change. Many components are deprecated and replaced with new ones. After looking at the hundreds of warning messages I gave up converting it. There are two new versions of Angular and PrimeNG every year and it could be a full-time job just to keep upgrading, so I have stuck to V16.
 +
 +
Newer versions of Angular and PrimeNG expect us to convert from module based to stand-alone components. That is supposed to save on memory use. We can do that now, but I have not. In future if we upgrade further we will have to change. I have tested it and stand alone components seem to be noticeably slower than module based. The memory used seemed to be similar so for this application there seems to be no advantage in using stand alone components.
 +
 +
If a package in package.json or package-lock.json is out of date, and there is a new one that is compatible with the angular and other packages in use, you can move to the newer version. To see what versions are compatible, see https://angular.dev/reference/versions .
 +
 +
Do not edit package-json or package-lock.json. Instead, in  mythtv/mythtv/html/backend, run
 +
 +
* npm install package-name@version
 +
 +
For example npm install typescript@4.8.5
 +
 +
You can get a list of package versions at https://www.npmjs.com/package/primeng?activeTab=versions. Search the package name and click the versions tab.
 +
 +
To upgrade Angular/cli major version, follow the steps at https://angular.dev/update-guide. There are utilities that modify our code to conform to the new version, and instructions for things to do manually. You will need to also upgrade PrimeNG. Unfortunately, there is no upgrade utility and you will have to manually fix anything that breaks as a result. Sometimes there are cross dependencies that prevent installing a newer version. In that case install the new versions with --force. Once all are installed, check the dependencies with npm audit.
 +
 +
After upgrading PrimeNG, update the themes that are in our source repository, by running update_themes.sh from the html/backend directory. Out of date themes can cause UI problems.
 +
 +
=== PrimeNG Versions ===
  
 +
PrimeNG major version must always match Angular major version. Do not select any PrimeNG version with the label '''lts'''. The lts versions are not free and require purchase of a license. Select the highest non-lts version of a particular major version that you are using.
  
−
== Getting started  ==
+
=== Visual Studio Code ===
−
The WebApp uses [https://angular.io/ Angular] and [https://www.primefaces.org/primeng/setup PrimeNG]
 
  
 
[https://code.visualstudio.com/ Visual Studio Code (VSC)] development environment is recommended to work on the WebApp.
 
[https://code.visualstudio.com/ Visual Studio Code (VSC)] development environment is recommended to work on the WebApp.
  
−
To load the code into VSC cd to /mythtv/http/mythbackend and run '''code .''' to start working on the webapp.
+
To load the code into VSC cd to /mythtv/html/backend and run '''code .''' to start working on the WebApp.
 +
 
 +
VSC has many useful extensions to aid development. Some useful ones are Lingua - Managing nxg-translations, Code Spell Checker and Trailing Spaces.
 +
 
 +
==== Starting the development web server ====
 +
 
 +
First start a local copy of mythbackend. Certain URLs will be sent to the backend to be answered. (see proxy.conf.js section for details)
 +
 
 +
In a terminal in VSC type '''npx ng serve''' to start the development http server and point your web browser to '''http://localhost:4200/''' (alternatively the shortcut '''npx ng serve --open'''). Each time you save something in VSC the dev web server causes your browser to reload the web app making it easy to make changes.
 +
 
 +
==== proxy.conf.js ====
 +
 
 +
The file '''mythtv/html/backend/src/proxy.conf.js''' controls how the development web server handles URLs. Without this configuration, all the API calls to the backend would fail, since they are not part of the url routes handled by the webapp.
 +
 
 +
You can change this to point to any backend you wish to serve the API from (eg. a backend with guide data). You may create additional stanzas to send different URLs to different backends should you wish.
 +
 
 +
This allows your development environment to be very flexible.
 +
 
 +
=== Translations ===
 +
 
 +
We use [http://www.ngx-translate.com/ ngx-translate] to handle the loading of translations which allows us to switch translations without reloading the webapp.
 +
 
 +
The translation files are simple json files which can be edited with any text editor or there are tools designed to create and edit them like [https://www.codeandweb.com/babeledit BabelEdit].
 +
 
 +
You can find the translation files in the /mythtv/html/assets/i18n/ folder.
 +
 
 +
If you are using VSC a good plugin to manage translations is [https://marketplace.visualstudio.com/items?itemName=chr33z.lingua-vscode Lingua - Managing nxg-translations]
 +
 
 +
==== How to make a string in an html file translatable ====
 +
 
 +
You simply use the translate pipe like this
 +
<pre><h1>{{ 'setupwizard.title' | translate }}</h1></pre>
 +
 
 +
and in the various translation files you would add
 +
 
 +
<pre>
 +
"setupwizard": {
 +
        "title": "Initial Setup Wizard",
 +
    }
 +
</pre>
 +
 
 +
If the string you want translated includes html, for example
 +
<pre>
 +
{
 +
    ...
 +
    "settings": {
 +
        ...
 +
        "hostaddress": {
 +
            "master_desc": "Host name of the Master Backend. <strong>Read only Setting</strong> This is set by selecting \"<i>This server is the Master Backend</i>\" on that server."
 +
        ...
 +
</pre>
 +
 
 +
then you need the following syntax, otherwise the html tags are displayed in the page instead of taking effect:
 +
 
 +
<pre>
 +
    <div [innerHTML]="'settings.hostaddress.master_desc' | translate"></div>
 +
</pre>
 +
 
 +
====How to use the Lingua VSC plugin to add a new translation in an html file====
 +
# Select the text you want to translate
 +
# Right click on it and choose '''Lingua: Convert text to translation'''.
 +
# Choose the language you want to add the translation to from the list.
 +
# Choose the key you want to use eg from the example above the key is '''setupwizard.title'''.
 +
# To check it was added OK right click it and choose '''Lingua: Go to translation''' - You may have to save the xxx.json file before trying to install them.
 +
# IMPORTANT: before the WebApp can see the new translation, even when using the Angular dev server, you must do a sudo make install in the html directory since it's the backend server that makes the translation files available to the WebApp.
 +
# If you want to test outside of the Angular dev server eg. you want to test your changes on  a real backend server you must do a '''npm run build''' to rebuild the WebApp and install that before you will be able to see your changes.
 +
 
 +
==== Creating default translations with google translate ====
 +
After adding entries to en_US.json, you can create google translations in the other files.
 +
 
 +
Install required dependencies:
 +
<pre>
 +
sudo pip3 install flatten-dict
 +
sudo pip3 install googletrans==3.1.0a0
 +
</pre>
 +
 
 +
There is a program translation-util.py in the mythtv/html/ directory in git. Change to that directory and run below command. This will translate new strings added but will not translate any existing strings that have been altered.
 +
<pre>
 +
./translation-util.py -t
 +
</pre>
 +
To get a list of other options, run
 +
<pre>
 +
./translation-util.py -h
 +
</pre>
 +
If you need to change an existing string, you will need to run the -c option. Removing a string required the -r option.
 +
 
 +
The -l option (pipe into less) is very useful to get the key and string value for existing values when  you need to change or remove something.
 +
<pre>
 +
./translation-util.py -l | less
 +
</pre>
 +
 
 +
=== Themes ===
 +
Themes in PrimeNG are simple css files that define the look of the components available.
 +
 +
Most of the free themes available for PrimeNG have been added and made available for users to choose from. There are currently 34 to choose from. The free themes aren't particularly exciting but they do provide a little choice of colour and light or dark themes.
  
−
In a terminal in VSC type '''ng serve''' to start the development http server and point your web browser to '''http://localhost:4200/'''. Each time you save something in VSC the dev web server cause your browser to reload the web app making it easy to make changes.
+
You can find the css and fonts for the themes in mythtv/html/assets/themes/ as well as the themes.json file that tells the WebApp which themes are available.
  
−
Be aware that we use Angular v13.x and many tutorials and examples on the web may not compile or work properly if they was written for earlier versions.
+
==== How to add a theme ====
 +
Use an existing css file for a theme as a template and edit as required then add an entry in the themes.json file for your new theme. You would also have to create an icon for the theme and place it into mythtv/html/images/themes/.  
  
−
Many examples on the web put their assets like images, data files, fonts, css files etc in the src/assets directory unfortunately because of the way our webapp has to be proxied to get it to work with our web server that does not work. Instead you can put your assets into the  mythtv/html/assets directory which is made available to the webapp once it is installed. This can lead to some head scratching if you don't make sure the assets are installed before you start the backend and the assets are in there own folder since the backend will only search for them the first time it is started.
+
=== Production ===
  
 +
When you are happy with the changes you have made, you should run '''npm run build''' which will create a production build of the webapp in '''mythtv/html/apps/backend'''. These build artifacts should be checked in. Ultimately they will be served by the backend when you visit the new webserver on the backend (currently runs on port 6544 by default)
  
 +
This is done so that the webapp does not need to be rebuilt at build time, and none of the npm modules are required on a build server.
  
−
== Build the Web App ==
+
The built package consists of large amounts of javascript all in one line of the file. It is identified to git as a binary file because there is nothing meaningful you can see in the file or that github can do with attempting to diff or merge the files.
 
   
 
   
−
cd into .../mythtv/html/backend and run  '''npm run-script build''' to build the project. The build artefacts will be stored in the ../apps/backend directory. These should be
+
== Possible Problems ==
−
committed, so that the backend webapp does not need rebuilding at build time.
+
 
 +
Many examples on the web put their assets like images, data files, fonts, css files etc in the src/assets directory unfortunately because of the way our webapp has to be proxied to get it to work with our web server that does not work. Instead you can put your assets into the  mythtv/html/assets directory which is made available to the webapp once it is installed. This can lead to some head scratching if you don't make sure the assets are installed before you start the backend and the assets are in there own folder since the backend will only search for them the first time it is started.
 +
 
 +
[[Category:Developer Documentation]]

Latest revision as of 17:06, 1 May 2025

Basic instructions for working on the new WebApp

Getting started

The WebApp uses Angular, PrimeNG, PrimeFlex and ngx-translate.

Angular is the web framework we use.

PrimeNG is an Angular UI Component Library providing various form controls like text edits, check boxes, radio buttons, sliders as well as tables, panels, overlays etc.

To see the Version 16 documentation for PrimeNG, use https://primeng-v16.pages.dev/installation . The documentation is no longer on the main PrimeNG page.

PrimeFlex is a lightweight responsive CSS utility library to accompany Prime UI libraries.

ngx-translate handles loading translation.

Development vs Production

We have intentionally checked in the production build artifacts. This is because installing the build requirements uses a method not permitted by the packaging process of Linus distributions. The "npm install" command is a quick and easy way to download and install into your directory all of the dependents. This cannot be run by a Linux distribution build because it accesses the internet.

Install Development Software

To check what version of Angular and PrimeNG the web app is using, look at mythtv/mythtv/html/backend/package.json. The Angular version to look for is Angular/cli. The major version of Angular/cli must match the major version of PrimeNG. The high level version number of all angular components should be the same.

To see what version of Node.js is needed, see the table at https://angular.dev/reference/versions. In that table and in package.json, version number ^x.y.z means any version x.*.* as long as it is at least x.y.z. ~x.y.z means any version x.y.* as long as it is at least x.y.z.

You can install nodejs and npm from your linux distribution package manager, or install it from the web. I prefer to install it from the web in order to have control of the version being used

Install node js into a private directory. This way you can run two versions at once if needed.

You now have node installed in $HOME/.nvm and entries in .bashrc that cause your version to run when you run node. You can type node at a command prompt and have an interactive javascript interpreter where you can test javascript commands.

You can install multiple versions of nodejs. Then use the command "nvm use nn" to switch the currently used version.

Install packages used by the web app:

  • cd mythtv/mythtv/html/backend
  • npm install

This installs all required packages into subdirectory node_modules. It uses the package.json and other .json files to determine which packages to install. It will install all the required dependencies and ensure you are using the same versions as other developers. Occasionally you may need to re-run this if additional dependencies are added or updated

Do not install angular globally, in case you need to have different angular versions for different project versions.

Do not install node, npm or related packages from the system repository (apt get or similar). If you have, uninstall them.

Upgrading versions

We are currently using Angular version 16 and PrimeNG version 16. I have attempted to upgrade to version 17 and then to version 18. PrimeNG version 17 has minor changes but some things were not working when i attempted it, notably the lazy load feature used in several pages was no longer working. This feature seems to break with each new version. PrimeNG version 18 is a major change. Many components are deprecated and replaced with new ones. After looking at the hundreds of warning messages I gave up converting it. There are two new versions of Angular and PrimeNG every year and it could be a full-time job just to keep upgrading, so I have stuck to V16.

Newer versions of Angular and PrimeNG expect us to convert from module based to stand-alone components. That is supposed to save on memory use. We can do that now, but I have not. In future if we upgrade further we will have to change. I have tested it and stand alone components seem to be noticeably slower than module based. The memory used seemed to be similar so for this application there seems to be no advantage in using stand alone components.

If a package in package.json or package-lock.json is out of date, and there is a new one that is compatible with the angular and other packages in use, you can move to the newer version. To see what versions are compatible, see https://angular.dev/reference/versions .

Do not edit package-json or package-lock.json. Instead, in mythtv/mythtv/html/backend, run

  • npm install package-name@version

For example npm install typescript@4.8.5

You can get a list of package versions at https://www.npmjs.com/package/primeng?activeTab=versions. Search the package name and click the versions tab.

To upgrade Angular/cli major version, follow the steps at https://angular.dev/update-guide. There are utilities that modify our code to conform to the new version, and instructions for things to do manually. You will need to also upgrade PrimeNG. Unfortunately, there is no upgrade utility and you will have to manually fix anything that breaks as a result. Sometimes there are cross dependencies that prevent installing a newer version. In that case install the new versions with --force. Once all are installed, check the dependencies with npm audit.

After upgrading PrimeNG, update the themes that are in our source repository, by running update_themes.sh from the html/backend directory. Out of date themes can cause UI problems.

PrimeNG Versions

PrimeNG major version must always match Angular major version. Do not select any PrimeNG version with the label lts. The lts versions are not free and require purchase of a license. Select the highest non-lts version of a particular major version that you are using.

Visual Studio Code

Visual Studio Code (VSC) development environment is recommended to work on the WebApp.

To load the code into VSC cd to /mythtv/html/backend and run code . to start working on the WebApp.

VSC has many useful extensions to aid development. Some useful ones are Lingua - Managing nxg-translations, Code Spell Checker and Trailing Spaces.

Starting the development web server

First start a local copy of mythbackend. Certain URLs will be sent to the backend to be answered. (see proxy.conf.js section for details)

In a terminal in VSC type npx ng serve to start the development http server and point your web browser to http://localhost:4200/ (alternatively the shortcut npx ng serve --open). Each time you save something in VSC the dev web server causes your browser to reload the web app making it easy to make changes.

proxy.conf.js

The file mythtv/html/backend/src/proxy.conf.js controls how the development web server handles URLs. Without this configuration, all the API calls to the backend would fail, since they are not part of the url routes handled by the webapp.

You can change this to point to any backend you wish to serve the API from (eg. a backend with guide data). You may create additional stanzas to send different URLs to different backends should you wish.

This allows your development environment to be very flexible.

Translations

We use ngx-translate to handle the loading of translations which allows us to switch translations without reloading the webapp.

The translation files are simple json files which can be edited with any text editor or there are tools designed to create and edit them like BabelEdit.

You can find the translation files in the /mythtv/html/assets/i18n/ folder.

If you are using VSC a good plugin to manage translations is Lingua - Managing nxg-translations

How to make a string in an html file translatable

You simply use the translate pipe like this 

<h1>{{ 'setupwizard.title' | translate }}</h1>

and in the various translation files you would add 

"setupwizard": {
        "title": "Initial Setup Wizard",
    }

If the string you want translated includes html, for example 

{
    ...
    "settings": {
        ...
        "hostaddress": {
            "master_desc": "Host name of the Master Backend. <strong>Read only Setting</strong> This is set by selecting \"<i>This server is the Master Backend</i>\" on that server."
        ...

then you need the following syntax, otherwise the html tags are displayed in the page instead of taking effect: 

    <div [innerHTML]="'settings.hostaddress.master_desc' | translate"></div>

How to use the Lingua VSC plugin to add a new translation in an html file

  1. Select the text you want to translate
  2. Right click on it and choose Lingua: Convert text to translation.
  3. Choose the language you want to add the translation to from the list.
  4. Choose the key you want to use eg from the example above the key is setupwizard.title.
  5. To check it was added OK right click it and choose Lingua: Go to translation - You may have to save the xxx.json file before trying to install them.
  6. IMPORTANT: before the WebApp can see the new translation, even when using the Angular dev server, you must do a sudo make install in the html directory since it's the backend server that makes the translation files available to the WebApp.
  7. If you want to test outside of the Angular dev server eg. you want to test your changes on a real backend server you must do a npm run build to rebuild the WebApp and install that before you will be able to see your changes.

Creating default translations with google translate

After adding entries to en_US.json, you can create google translations in the other files.

Install required dependencies: 

sudo pip3 install flatten-dict
sudo pip3 install googletrans==3.1.0a0

There is a program translation-util.py in the mythtv/html/ directory in git. Change to that directory and run below command. This will translate new strings added but will not translate any existing strings that have been altered. 

./translation-util.py -t

To get a list of other options, run 

./translation-util.py -h

If you need to change an existing string, you will need to run the -c option. Removing a string required the -r option.

The -l option (pipe into less) is very useful to get the key and string value for existing values when you need to change or remove something. 

./translation-util.py -l | less

Themes

Themes in PrimeNG are simple css files that define the look of the components available.

Most of the free themes available for PrimeNG have been added and made available for users to choose from. There are currently 34 to choose from. The free themes aren't particularly exciting but they do provide a little choice of colour and light or dark themes.

You can find the css and fonts for the themes in mythtv/html/assets/themes/ as well as the themes.json file that tells the WebApp which themes are available.

How to add a theme

Use an existing css file for a theme as a template and edit as required then add an entry in the themes.json file for your new theme. You would also have to create an icon for the theme and place it into mythtv/html/images/themes/.

Production

When you are happy with the changes you have made, you should run npm run build which will create a production build of the webapp in mythtv/html/apps/backend. These build artifacts should be checked in. Ultimately they will be served by the backend when you visit the new webserver on the backend (currently runs on port 6544 by default)

This is done so that the webapp does not need to be rebuilt at build time, and none of the npm modules are required on a build server.

The built package consists of large amounts of javascript all in one line of the file. It is identified to git as a binary file because there is nothing meaningful you can see in the file or that github can do with attempting to diff or merge the files.

Possible Problems

Many examples on the web put their assets like images, data files, fonts, css files etc in the src/assets directory unfortunately because of the way our webapp has to be proxied to get it to work with our web server that does not work. Instead you can put your assets into the mythtv/html/assets directory which is made available to the webapp once it is installed. This can lead to some head scratching if you don't make sure the assets are installed before you start the backend and the assets are in there own folder since the backend will only search for them the first time it is started.

Retrieved from "http://www.mythtv.org/wiki?title=Web_Application_Development&oldid=66897"