kenmirrors/ente
1
0
Fork 0
mirror of https://github.com/ente-io/ente.git synced 2025-05-02 04:05:42 +00:00
Code Issues Packages Projects Releases Wiki Activity

[docs] Refactor

Browse Source
This commit is contained in:
vishnukvmd 2025-04-26 16:44:55 +05:30
parent 5bf3f01de6
commit 7098e93ae8
7 changed files with 157 additions and 147 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

19
docs/docs/.vitepress/sidebar.ts
View File

@ -224,6 +224,7 @@ export const sidebar = [
}, },
{ {
text: "Troubleshooting", text: "Troubleshooting",
collapsed: true,
items: [ items: [
{ {
text: "Windows login", text: "Windows login",
@ -239,25 +240,25 @@ export const sidebar = [
items: [ items: [
{ text: "Getting started", link: "/self-hosting/" }, { text: "Getting started", link: "/self-hosting/" },
{ {
text: "Connect to custom server", text: "Connecting to custom server",
link: "/self-hosting/guides/custom-server/", link: "/self-hosting/guides/custom-server/",
}, },
{ {
text: "Creating Accounts", text: "Creating accounts",
link: "/self-hosting/creating-accounts", link: "/self-hosting/creating-accounts",
}, },
{ {
text: "Reverse Proxy", text: "Configuring your server",
link: "/self-hosting/reverse-proxy",
},
{
text: "Building your museum.yaml",
link: "/self-hosting/museum", link: "/self-hosting/museum",
}, },
{ {
text: "Configuring S3", text: "Configuring S3",
link: "/self-hosting/guides/configuring-s3", link: "/self-hosting/guides/configuring-s3",
}, },
{
text: "Reverse proxy",
link: "/self-hosting/reverse-proxy",
},
{ {
text: "Guides", text: "Guides",
collapsed: true, collapsed: true,
@ -276,7 +277,7 @@ export const sidebar = [
link: "/self-hosting/guides/admin", link: "/self-hosting/guides/admin",
}, },
{ {
text: "Configure CLI for Self Hosted Instance", text: "Configure CLI for self hosted instance",
link: "/self-hosting/guides/selfhost-cli", link: "/self-hosting/guides/selfhost-cli",
}, },
{ {
@ -313,7 +314,7 @@ export const sidebar = [
link: "/self-hosting/troubleshooting/yarn", link: "/self-hosting/troubleshooting/yarn",
}, },
{ {
text: "Ente CLI Secrets", text: "Ente CLI secrets",
link: "/self-hosting/troubleshooting/keyring", link: "/self-hosting/troubleshooting/keyring",
}, },
], ],

BIN
docs/docs/public/web-app.webp Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 75 KiB

4
docs/docs/self-hosting/creating-accounts.md
View File

@ -1,9 +1,9 @@
--- ---
title: Creating Accounts title: Creating accounts
description: Creating accounts on your deployment description: Creating accounts on your deployment
--- ---
# Creating Accounts # Creating accounts
Once the docker containers are up and running on their desired ports. The Ente Photos Once the docker containers are up and running on their desired ports. The Ente Photos
web app will be accessible on `http://localhost:3000`. Open the URL in your browser web app will be accessible on `http://localhost:3000`. Open the URL in your browser

135
docs/docs/self-hosting/guides/configuring-s3.md
View File

@ -5,23 +5,26 @@ description:
from outside localhost from outside localhost
--- ---
# Components of the Architecture # Architecture
![Client, Museum, S3](/client-museum-s3.png) ![Client, Museum, S3](/client-museum-s3.png)
There are three components involved in uploading: There are three components involved in uploading a file:
1. The client (e.g. the web app or the mobile app) 1. The client (e.g. the web app or the mobile app)
2. Ente's server (museum) 2. Ente's server (museum)
3. The S3-compatible object storage (e.g. minio in the default starter) 3. The S3-compatible object storage (e.g. MinIO in the default starter)
For the uploads to work, all three of them need to be able to reach each other. For the uploads to work, all three of them need to be able to reach each other.
This is because the client uploads directly to the object storage. The This is because the client uploads directly to the object storage.
interaction goes something like this:
1. Client wants to upload, it asks museum where it should upload to. A file upload flows as follows:
2. Museum creates pre-signed URLs for the S3 bucket that was configured.
3. Client directly uploads to the S3 buckets these URLs. 1. Client that wants to upload a file asks museum where it should upload the
file to
2. museum creates pre-signed URLs for the S3 bucket that was configured
3. Client directly uploads to the S3 buckets these URLs
4. Client finally informs museum that a file has been uploaded to this URL
The upshot of this is that _both_ the client and museum should be able to reach The upshot of this is that _both_ the client and museum should be able to reach
your S3 bucket. your S3 bucket.
@ -30,10 +33,10 @@ your S3 bucket.
The URL for the S3 bucket is configured in The URL for the S3 bucket is configured in
[scripts/compose/credentials.yaml](https://github.com/ente-io/ente/blob/main/server/scripts/compose/credentials.yaml#L10). [scripts/compose/credentials.yaml](https://github.com/ente-io/ente/blob/main/server/scripts/compose/credentials.yaml#L10).
You can edit this file directly when testing, though it is just simpler and more
robust to create a `museum.yaml` (in the same folder as the Docker compose file) You can edit this file directly while testing, though it is more robust to
and put your custom configuration there (in your case, you can put an entire create a `museum.yaml` (in the same folder as the Docker compose file) and to
`s3` config object in your `museum.yaml`). setup your custom configuration there.
> [!TIP] > [!TIP]
> For more details about these configuration objects, see the documentation for > For more details about these configuration objects, see the documentation for
@ -42,29 +45,32 @@ and put your custom configuration there (in your case, you can put an entire
By default, you only need to configure the endpoint for the first bucket. By default, you only need to configure the endpoint for the first bucket.
The docker compose file is shipped with MinIO as the Self Hosted S3 Compatible Storage. The Docker compose file is shipped with MinIO as the self hosted S3 compatible
By default, MinIO server is served on `localhost:3200` and the MinIO UI on storage. By default, MinIO server is served on `localhost:3200` and the MinIO UI
`localhost:3201`. on `localhost:3201`.
For example, in a localhost network situation, the way this
connection works is, Museum (`1`) and MinIO (`2`) run on the same docker network and For example, in a localhost network situation, the way this connection works is,
the web app (`3`) which will also be hosted on the localhost. This enables all the museum (`1`) and MinIO (`2`) run on the same Docker network and the web app
three components of the setup being able to communicate with each other seamlessly. (`3`) will also be hosted on your localhost. This enables all the three
components of the setup to communicate with each other seamlessly.
The same principle applies if you're deploying to your custom domain. The same principle applies if you're deploying to your custom domain.
## Replication ## Replication
![Replication](/replication.png) ![Replication](/replication.png)
<p align="center">Community contributed diagram of Ente's Replication Process</p> <p align="center">Community contributed diagram of Ente's replication process</p>
> [!IMPORTANT] > [!IMPORTANT]
> As of now, Replication works only if all the 3 storage type >
> needs are fulfilled (1 Hot, 1 Cold and 1 Glacier Storage). > As of now, replication works only if all the 3 storage type needs are
> fulfilled (1 hot, 1 cold and 1 glacier storage).
> >
> [Reference](https://github.com/ente-io/ente/discussions/3167#discussioncomment-10585970) > [Reference](https://github.com/ente-io/ente/discussions/3167#discussioncomment-10585970)
If you're wondering why there are 3 buckets on MinIO UI - that's because our If you're wondering why there are 3 buckets on the MinIO UI - that's because our
production instance uses these to perform [replication](https://ente.io/reliability/). production instance uses these to perform
[replication](https://ente.io/reliability/).
If you're also wondering about why the bucket names are specifically what they are, If you're also wondering about why the bucket names are specifically what they are,
it's because that is exactly what we are using on our production instance. it's because that is exactly what we are using on our production instance.
@ -72,10 +78,10 @@ We use `b2-eu-cen` as hot, `wasabi-eu-central-2-v3` as cold (also the secondary
and `scw-eu-fr-v3` as glacier storage. As of now, all of this is hardcoded. and `scw-eu-fr-v3` as glacier storage. As of now, all of this is hardcoded.
Hence, the same hardcoded configuration is applied when you self host Ente. Hence, the same hardcoded configuration is applied when you self host Ente.
In a Self hosted Ente Instance replication is turned off by default. In a self hosted Ente instance replication is turned off by default. When
When replication is turned off, only the first bucket (`b2-eu-cen`) is used, replication is turned off, only the first bucket (`b2-eu-cen`) is used, and the
and the other two are ignored. Only the names here are specifically fixed, but other two are ignored. Only the names here are specifically fixed, but in the
in the configuration body you can put any other keys. It does not have any relation configuration body you can put any other keys. It does not have any relation
with `b2`, `wasabi` or even `scaleway`. with `b2`, `wasabi` or even `scaleway`.
Use the `s3.hot_storage.primary` option if you'd like to set one of the other Use the `s3.hot_storage.primary` option if you'd like to set one of the other
@ -85,23 +91,23 @@ predefined buckets as the primary bucket.
> [!NOTE] > [!NOTE]
> >
> If you need to configure SSL, for example if you're running over the internet, > If you need to configure SSL, you'll need to turn off `s3.are_local_buckets`
> you'll need to turn off `s3.are_local_buckets` (which disables SSL in the > (which disables SSL in the default starter compose template).
> default starter compose template).
> >
Disabling `s3.are_local_buckets` also switches to the subdomain style URLs for Disabling `s3.are_local_buckets` also switches to the subdomain style URLs for
the buckets. However, not all S3 providers support these, in particular, minio the buckets. However, not all S3 providers support these. In particular, MinIO
does not work with these in default configuration. So in such cases you'll does not work with these in default configuration. So in such cases you'll also
also need to then enable `s3.use_path_style_urls`. need to enable `s3.use_path_style_urls`.
## Summary ## Summary
Set the S3 bucket `endpoint` in `credentials.yaml` to a `yourserverip:3200` or Set the S3 bucket `endpoint` in `credentials.yaml` to a `yourserverip:3200` or
some such IP/hostname that accessible from both where you are running the Ente some such IP / hostname that is accessible from both where you are running the
clients (e.g. the mobile app) and also from within the Docker compose cluster. Ente clients (e.g. the mobile app) and also from within the Docker compose
cluster.
#### Example ### Example
An example `museum.yaml` when you're trying to connect to museum running on your An example `museum.yaml` when you're trying to connect to museum running on your
computer from your phone on the same WiFi network: computer from your phone on the same WiFi network:
@ -117,49 +123,44 @@ s3:
bucket: b2-eu-cen bucket: b2-eu-cen
``` ```
## FAE (Frequently Answered Errors) ## Frequently encountered errors
Here are some Frequently Answered Errors from the Community Chat with the reasoning Here are some errors our community members frequently encountered with the
for a particular error and its potential fix. context and potential fixes.
In most situations, the problem turns out to be some minute mistakes or misconfigurations In most situations, the problem is because of a minor mistakes or
on the users end and that turns out to be the bottleneck of the whole problem. misconfiguration. Please make sure to `reverse_proxy` museum to a domain and
Please make sure to `reverse_proxy` Museum to a domain as well as check your S3 check your S3 credentials and whole configuration file for any minor
Credentials and whole config for any minor mis-configurations. misconfigurations.
It is also suggested that the user setups Bucket CORS on MinIO or any external It is also suggested that the user setups bucket CORS on MinIO or any external
S3 Bucket they are connecting to. To setup Bucket CORS, help yourself by upload S3 service provider they are connecting to. To setup bucket CORS, please [read
[this](https://help.ente.io/self-hosting/guides/external-s3#_5-fix-potential-cors-issue-with-your-bucket). this](/self-hosting/guides/external-s3#_5-fix-potential-cors-issue-with-your-bucket).
### 403 Forbidden ### 403 Forbidden
If museum (`2`) is able to make a network connection to your S3 bucket (`3`) but If museum (`2`) is able to make a network connection to your S3 bucket (`3`) but
uploads are still failing, it could be a credentials or permissions issue. A uploads are still failing, it could be a credentials or permissions issue.
telltale sign of this is that in the museum logs you can see `403 Forbidden`
A telltale sign of this is that in the museum logs you can see `403 Forbidden`
errors about it not able to find the size of a file even though the errors about it not able to find the size of a file even though the
corresponding object exists in the S3 bucket. corresponding object exists in the S3 bucket.
To fix these, you should ensure the following: This could be because
1. The bucket CORS rules do not allow museum to access these objects. 1. The bucket CORS rules do not allow museum to access these objects. For
- For uploading files from the browser, you will need to currently set uploading files from the browser, you will need to set `allowedOrigins` to
allowedOrigins to "\*", and allow the "X-Auth-Token", "X-Client-Package" `*`, and allow the `X-Auth-Token`, `X-Client-Package` headers configuration
headers configuration too. too. [Here is an example of a working
[Here is an example of a working configuration](https://github.com/ente-io/ente/discussions/1764#discussioncomment-9478204). configuration](https://github.com/ente-io/ente/discussions/1764#discussioncomment-9478204).
2. The credentials are not being picked up (you might be setting the correct 2. The credentials are not being picked up (you might be setting the correct
creds, but not in the place where museum picks them from). credentials, but not in the place where museum reads them from).
### Mismatch in File Size ### Mismatch in file size
The "Mismatch in File Size" error mostly occurs in a situation where the client (`1`) The "Mismatch in file size" error mostly occurs in a situation where the client
is re-uploading a file which is already in the bucket with a different File Size. The (`1`) is re-uploading a file which is already in the bucket with a different
reason for re-upload could be anything including Network issue, sudden killing of app file size. The reason for re-upload could be anything including network issue,
before the upload is complete and etc. sudden killing of app before the upload is complete and etc.
This is also one of Museums (`2`) Validation Checks for the size of file being
re-uploaded from the client to the size of the file which is already
uploaded to the S3 Bucket.
In most case, it is very unlikely that this error could be a cause of some mistake in
the configuration or Browser/Bucket CORS.

30
docs/docs/self-hosting/index.md
View File

@ -10,34 +10,32 @@ the same code we use for our own cloud service.
> [!TIP] > [!TIP]
> >
> To get some context, you might find our > You might find our [blog post](https://ente.io/blog/open-sourcing-our-server/)
> [blog post](https://ente.io/blog/open-sourcing-our-server/) announcing the > announcing the open sourcing of our server useful.
> open sourcing of our server useful.
## System requirements
## System Requirements The server has minimal resource requirements, running as a lightweight Go
binary. It performs well on small cloud instances, old laptops, and even
The server has minimal resource requirements, running as a lightweight Go binary [low-end embedded devices](https://github.com/ente-io/ente/discussions/594).
with no server-side ML. It performs well on small cloud instances, old laptops,
and even [low-end embedded devices](https://github.com/ente-io/ente/discussions/594)
reported by community members. Virtually any reasonable hardware should be sufficient.
## Getting started ## Getting started
Execute the below one-liner command in your terminal to setup Ente on your system. Run this command on your terminal to setup Ente.
```sh ```sh
sh -c "$(curl -fsSL https://raw.githubusercontent.com/ente-io/ente/main/server/quickstart.sh)" sh -c "$(curl -fsSL https://raw.githubusercontent.com/ente-io/ente/main/server/quickstart.sh)"
``` ```
The above `curl` command is a simple shell-script, which pulls the docker images, The above `curl` command pulls the Docker image, creates a directory `my-ente`
creates a directory `my-ente` in the current working directory and starts all the in the current working directory and starts all containers required to run Ente.
containers required to run Ente on your system.
![quickstart](/quickstart.png) ![quickstart](/quickstart.png)
![self-hosted-ente](/web-app.webp)
## Queries? ## Queries?
If you need any help or support, do not hesitate to drop your queries on our community If you need support, please ask on our community
[discord channel](https://discord.gg/z2YVKkycX3) or create a [Discord](https://ente.io/discord) or start a discussion on
[Github Discussion](https://github.com/ente-io/ente/discussions) where 100s of self-hosters help each other. [GitHub](https://github.com/ente-io/ente/discussions/).

75
docs/docs/self-hosting/museum.md
View File

@ -1,41 +1,49 @@
--- ---
title: Building your Museum.yaml title: Configuring your server
description: Guide to writing a museum.yaml description: Guide to writing a museum.yaml
--- ---
## Configuring `museum.yaml` # Configuring your server
`Museum.yaml` is a YAML configuration file used to configure various things for museum. Ente's monolithic server is called **museum**.
By default, [`local.yaml`](https://github.com/ente-io/ente/tree/main/server/configurations/local.yaml)
is also available, but it is overridden if `museum.yaml` file is found. We highly `museum.yaml` is a YAML configuration file used to configure museum. By default,
recommend creating and building your own `museum.yaml` instead of editing `configurations/local.yaml`. [`local.yaml`](https://github.com/ente-io/ente/tree/main/server/configurations/local.yaml)
The `my-ente` directory will include a `museum.yaml` file with some configurations around encryption is provided, but its settings are overridden with those from `museum.yaml`.
keys and secrets, postgres DB, and MinIO.
If you used our quickstart script, your `my-ente` directory will include a
`museum.yaml` file with preset configurations for encryption keys, secrets,
PostgreSQL and MinIO.
> [!TIP] > [!TIP]
> Always do `docker compose down` inside `my-ente` directory, if you've made any changes to `museum.yaml` >
> and then restart the containers with `docker compose up -d ` to see the changes in action. > Always do `docker compose down` inside your `my-ente` directory. If you've
> made changes to `museum.yaml`, restart the containers with `docker compose up
> -d ` to see your changes in action.
### S3 Buckets ## S3 buckets
By default, the `s3` section is configured to use local minIO buckets and for the same reason The `s3` section within `museum.yaml` is by default configured to use local
`are_local_buckets` is set to `true`. If you wish to bring any external S3 provider, MinIO buckets.
you just have to edit the configuration with appropriate credentails and details given by the provider.
And set `are_local_buckets` to false. Check out [Configuring S3](/self-hosting/guides/configuring-s3.md)
to understand more on how to configure S3 buckets and how the communication happens.
MinIO makes use of the port `3200` for API Endpoints and the Client Web App is run over `:3201` If you wish to use an external S3 provider, you can edit the configuration with
(both on localhost). You can login to MinIO Console Web UI by accessing `localhost:3201` in your web-browser your provider's credentials, and set `are_local_buckets` to `false`.
and setting up all the things related to regions there itself.
If you face any issues related to uploads then checkout Check out [Configuring S3](/self-hosting/guides/configuring-s3.md) to understand
[Troubleshooting Bucket CORS](/self-hosting/troubleshooting/bucket-cors) and more about configuring S3 buckets.
[Frequently Answered Error related to S3](/self-hosting/guides/configuring-s3#fae-frequently-answered-errors)
### App Endpoints MinIO uses the port `3200` for API Endpoints and their web app runs over
`:3201`. You can login to MinIO Web Console by opening `localhost:3201` in your browser.
Ente Photos Web app is divided into multiple sub-apps like albums, cast, auth, etc. If you face any issues related to uploads then checkout [Troubleshooting bucket
These endpoints are configurable in the museum.yaml under the `apps.*` section. CORS](/self-hosting/troubleshooting/bucket-cors) and [Frequently encountered S3
errors](/self-hosting/guides/configuring-s3#frequently-encountered-errors).
## App endpoints
Ente Photos Web app is divided into multiple sub-apps like albums, cast, auth,
etc. These endpoints are configurable in the museum.yaml under the `apps.*`
section.
For example, For example,
@ -48,13 +56,14 @@ apps:
``` ```
By default, all the values redirect to our publicly hosted production services. By default, all the values redirect to our publicly hosted production services.
After you are done with filling the values, restart museum and the App will start utilizing After you are done with filling the values, restart museum and the app will
those endpoints for everything instead of the Ente's prod instances. start utilizing those endpoints instead of Ente's production instances.
Once you configure all the necessary endpoints, `cd` into `my-ente` and stop all the docker Once you have configured all the necessary endpoints, `cd` into `my-ente` and
containers with `docker compose down` to completely stop all the containers and restart them stop all the Docker containers with `docker compose down` and restart them with
with `docker compose up -d`. `docker compose up -d`.
Similarly, you can read the default [`local.yaml`](https://github.com/ente-io/ente/tree/main/server/configurations/local.yaml) Similarly, you can use the default
and build a functioning `museum.yaml` for many other functionalities like SMTP, Discord [`local.yaml`](https://github.com/ente-io/ente/tree/main/server/configurations/local.yaml)
Notifications, Hardcoded-OTT's, etc. as a reference for building a functioning `museum.yaml` for many other
functionalities like SMTP, Discord notifications, Hardcoded-OTTs, etc.

39
docs/docs/self-hosting/reverse-proxy.md
View File

@ -3,35 +3,34 @@ Title: Configuring Reverse Proxy
Description: configuring reverse proxy for Museum and other endpoints Description: configuring reverse proxy for Museum and other endpoints
--- ---
## Reverse Proxy # Reverse proxy
This step isn't really the direct next step after creating an account. It is Ente's server (museum) runs on port `:8080`, web app on `:3000` and the other
one of the most essential steps to avoid certain CORS errors and will help you through apps from ports `3001-3004`.
the configuration coming ahead.
Museum runs on port `:8080`, Ente Photos web app runs on `:3000` and so on the other apps We highly recommend using HTTPS for Museum (`8080`). For security reasons museum
are lined up after each other from ports `3001-3004`. will not accept incoming HTTP traffic.
We highly recommend using HTTPS for Museum (`8080`). Primarily, because for security reasons Museum Head over to your DNS management dashboard and setup the appropriate records for
won't accept any incoming HTTP traffic. Hence, all the requests will fail. the endpoints. Mostly, `A` or `AAAA` records targeting towards your server's IP
address should be sufficient. The rest of the work will be done by the web
Head over to your DNS Management Dashboard and setup the appropriate records for the endpoints. server on your machine.
Mostly, `A` or `AAAA` records targeting towards your server's IP address should be sufficient. The rest of the work
will be done by the web server sitting on your server machine.
![cloudflare](/cloudflare.png) ![cloudflare](/cloudflare.png)
### With Caddy ### Caddy
Setting up a reverse proxy with Caddy is pretty easy and straightforward. Firstly, install Caddy Setting up a reverse proxy with Caddy is easy and straightforward.
on your server machine.
Firstly, install Caddy on your server.
```sh ```sh
sudo apt install caddy sudo apt install caddy
``` ```
After the installation is complete, a `Caddyfile` is created on the path `/etc/caddy/`. This file is After the installation is complete, a `Caddyfile` is created on the path
used to configure reverse proxies and a whole lot of different things. `/etc/caddy/`. This file is used to configure reverse proxies among other
things.
```yaml ```yaml
# Caddyfile - myente.xyz is just an example. # Caddyfile - myente.xyz is just an example.
@ -44,5 +43,7 @@ ente.myente.xyz {
#...and so on for other endpoints #...and so on for other endpoints
``` ```
After a few hard-reloads, Ente Photos web app should be up on https://ente.myente.xyz. You can check out After a hard-reload, the Ente Photos web app should be up on https://ente.myente.xyz.
the documentation for any other reverse proxy tool (like nginx) you want to use.
If you are using a different tool for reverse proxy (like nginx), please check
out their documentation.