kenmirrors/ente
1
0
Fork 0
mirror of https://github.com/ente-io/ente.git synced 2025-04-29 19:15:33 +00:00
Code Issues Packages Projects Releases Wiki Activity

[docs] Refactor

Browse Source
This commit is contained in:
vishnukvmd 2025-04-26 16:44:55 +05:30
parent 5bf3f01de6
commit 7098e93ae8
7 changed files with 157 additions and 147 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

19
docs/docs/.vitepress/sidebar.ts
View File

@ -224,6 +224,7 @@ export const sidebar = [
},
{
text: "Troubleshooting",
collapsed: true,
items: [
{
text: "Windows login",
@ -239,25 +240,25 @@ export const sidebar = [
items: [
{ text: "Getting started", link: "/self-hosting/" },
{
text: "Connect to custom server",
text: "Connecting to custom server",
link: "/self-hosting/guides/custom-server/",
},
{
text: "Creating Accounts",
text: "Creating accounts",
link: "/self-hosting/creating-accounts",
},
{
text: "Reverse Proxy",
link: "/self-hosting/reverse-proxy",
},
{
text: "Building your museum.yaml",
text: "Configuring your server",
link: "/self-hosting/museum",
},
{
text: "Configuring S3",
link: "/self-hosting/guides/configuring-s3",
},
{
text: "Reverse proxy",
link: "/self-hosting/reverse-proxy",
},
{
text: "Guides",
collapsed: true,
@ -276,7 +277,7 @@ export const sidebar = [
link: "/self-hosting/guides/admin",
},
{
text: "Configure CLI for Self Hosted Instance",
text: "Configure CLI for self hosted instance",
link: "/self-hosting/guides/selfhost-cli",
},
{
@ -313,7 +314,7 @@ export const sidebar = [
link: "/self-hosting/troubleshooting/yarn",
},
{
text: "Ente CLI Secrets",
text: "Ente CLI secrets",
link: "/self-hosting/troubleshooting/keyring",
},
],

BIN
docs/docs/public/web-app.webp Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 75 KiB

4
docs/docs/self-hosting/creating-accounts.md
View File

@ -1,9 +1,9 @@
---
title: Creating Accounts
title: Creating accounts
description: Creating accounts on your deployment
---
# Creating Accounts
# Creating accounts
Once the docker containers are up and running on their desired ports. The Ente Photos
web app will be accessible on `http://localhost:3000`. Open the URL in your browser

135
docs/docs/self-hosting/guides/configuring-s3.md
View File

@ -5,23 +5,26 @@ description:
from outside localhost
---
# Components of the Architecture
# Architecture
![Client, Museum, S3](/client-museum-s3.png)
There are three components involved in uploading:
There are three components involved in uploading a file:
1. The client (e.g. the web app or the mobile app)
2. Ente's server (museum)
3. The S3-compatible object storage (e.g. minio in the default starter)
3. The S3-compatible object storage (e.g. MinIO in the default starter)
For the uploads to work, all three of them need to be able to reach each other.
This is because the client uploads directly to the object storage. The
interaction goes something like this:
This is because the client uploads directly to the object storage.
1. Client wants to upload, it asks museum where it should upload to.
2. Museum creates pre-signed URLs for the S3 bucket that was configured.
3. Client directly uploads to the S3 buckets these URLs.
A file upload flows as follows:
1. Client that wants to upload a file asks museum where it should upload the
file to
2. museum creates pre-signed URLs for the S3 bucket that was configured
3. Client directly uploads to the S3 buckets these URLs
4. Client finally informs museum that a file has been uploaded to this URL
The upshot of this is that _both_ the client and museum should be able to reach
your S3 bucket.
@ -30,10 +33,10 @@ your S3 bucket.
The URL for the S3 bucket is configured in
[scripts/compose/credentials.yaml](https://github.com/ente-io/ente/blob/main/server/scripts/compose/credentials.yaml#L10).
You can edit this file directly when testing, though it is just simpler and more
robust to create a `museum.yaml` (in the same folder as the Docker compose file)
and put your custom configuration there (in your case, you can put an entire
`s3` config object in your `museum.yaml`).
You can edit this file directly while testing, though it is more robust to
create a `museum.yaml` (in the same folder as the Docker compose file) and to
setup your custom configuration there.
> [!TIP]
> For more details about these configuration objects, see the documentation for
@ -42,29 +45,32 @@ and put your custom configuration there (in your case, you can put an entire
By default, you only need to configure the endpoint for the first bucket.
The docker compose file is shipped with MinIO as the Self Hosted S3 Compatible Storage.
By default, MinIO server is served on `localhost:3200` and the MinIO UI on
`localhost:3201`.
For example, in a localhost network situation, the way this
connection works is, Museum (`1`) and MinIO (`2`) run on the same docker network and
the web app (`3`) which will also be hosted on the localhost. This enables all the
three components of the setup being able to communicate with each other seamlessly.
The Docker compose file is shipped with MinIO as the self hosted S3 compatible
storage. By default, MinIO server is served on `localhost:3200` and the MinIO UI
on `localhost:3201`.
For example, in a localhost network situation, the way this connection works is,
museum (`1`) and MinIO (`2`) run on the same Docker network and the web app
(`3`) will also be hosted on your localhost. This enables all the three
components of the setup to communicate with each other seamlessly.
The same principle applies if you're deploying to your custom domain.
## Replication
![Replication](/replication.png)
<p align="center">Community contributed diagram of Ente's Replication Process</p>
<p align="center">Community contributed diagram of Ente's replication process</p>
> [!IMPORTANT]
> As of now, Replication works only if all the 3 storage type
> needs are fulfilled (1 Hot, 1 Cold and 1 Glacier Storage).
>
> As of now, replication works only if all the 3 storage type needs are
> fulfilled (1 hot, 1 cold and 1 glacier storage).
>
> [Reference](https://github.com/ente-io/ente/discussions/3167#discussioncomment-10585970)
If you're wondering why there are 3 buckets on MinIO UI - that's because our
production instance uses these to perform [replication](https://ente.io/reliability/).
If you're wondering why there are 3 buckets on the MinIO UI - that's because our
production instance uses these to perform
[replication](https://ente.io/reliability/).
If you're also wondering about why the bucket names are specifically what they are,
it's because that is exactly what we are using on our production instance.
@ -72,10 +78,10 @@ We use `b2-eu-cen` as hot, `wasabi-eu-central-2-v3` as cold (also the secondary
and `scw-eu-fr-v3` as glacier storage. As of now, all of this is hardcoded.
Hence, the same hardcoded configuration is applied when you self host Ente.
In a Self hosted Ente Instance replication is turned off by default.
When replication is turned off, only the first bucket (`b2-eu-cen`) is used,
and the other two are ignored. Only the names here are specifically fixed, but
in the configuration body you can put any other keys. It does not have any relation
In a self hosted Ente instance replication is turned off by default. When
replication is turned off, only the first bucket (`b2-eu-cen`) is used, and the
other two are ignored. Only the names here are specifically fixed, but in the
configuration body you can put any other keys. It does not have any relation
with `b2`, `wasabi` or even `scaleway`.
Use the `s3.hot_storage.primary` option if you'd like to set one of the other
@ -85,23 +91,23 @@ predefined buckets as the primary bucket.
> [!NOTE]
>
> If you need to configure SSL, for example if you're running over the internet,
> you'll need to turn off `s3.are_local_buckets` (which disables SSL in the
> default starter compose template).
> If you need to configure SSL, you'll need to turn off `s3.are_local_buckets`
> (which disables SSL in the default starter compose template).
>
Disabling `s3.are_local_buckets` also switches to the subdomain style URLs for
the buckets. However, not all S3 providers support these, in particular, minio
does not work with these in default configuration. So in such cases you'll
also need to then enable `s3.use_path_style_urls`.
the buckets. However, not all S3 providers support these. In particular, MinIO
does not work with these in default configuration. So in such cases you'll also
need to enable `s3.use_path_style_urls`.
## Summary
Set the S3 bucket `endpoint` in `credentials.yaml` to a `yourserverip:3200` or
some such IP/hostname that accessible from both where you are running the Ente
clients (e.g. the mobile app) and also from within the Docker compose cluster.
some such IP / hostname that is accessible from both where you are running the
Ente clients (e.g. the mobile app) and also from within the Docker compose
cluster.
#### Example
### Example
An example `museum.yaml` when you're trying to connect to museum running on your
computer from your phone on the same WiFi network:
@ -117,49 +123,44 @@ s3:
bucket: b2-eu-cen
```
## FAE (Frequently Answered Errors)
## Frequently encountered errors
Here are some Frequently Answered Errors from the Community Chat with the reasoning
for a particular error and its potential fix.
Here are some errors our community members frequently encountered with the
context and potential fixes.
In most situations, the problem turns out to be some minute mistakes or misconfigurations
on the users end and that turns out to be the bottleneck of the whole problem.
Please make sure to `reverse_proxy` Museum to a domain as well as check your S3
Credentials and whole config for any minor mis-configurations.
In most situations, the problem is because of a minor mistakes or
misconfiguration. Please make sure to `reverse_proxy` museum to a domain and
check your S3 credentials and whole configuration file for any minor
misconfigurations.
It is also suggested that the user setups Bucket CORS on MinIO or any external
S3 Bucket they are connecting to. To setup Bucket CORS, help yourself by upload
[this](https://help.ente.io/self-hosting/guides/external-s3#_5-fix-potential-cors-issue-with-your-bucket).
It is also suggested that the user setups bucket CORS on MinIO or any external
S3 service provider they are connecting to. To setup bucket CORS, please [read
this](/self-hosting/guides/external-s3#_5-fix-potential-cors-issue-with-your-bucket).
### 403 Forbidden
If museum (`2`) is able to make a network connection to your S3 bucket (`3`) but
uploads are still failing, it could be a credentials or permissions issue. A
telltale sign of this is that in the museum logs you can see `403 Forbidden`
uploads are still failing, it could be a credentials or permissions issue.
A telltale sign of this is that in the museum logs you can see `403 Forbidden`
errors about it not able to find the size of a file even though the
corresponding object exists in the S3 bucket.
To fix these, you should ensure the following:
This could be because
1. The bucket CORS rules do not allow museum to access these objects.
- For uploading files from the browser, you will need to currently set
allowedOrigins to "\*", and allow the "X-Auth-Token", "X-Client-Package"
headers configuration too.
[Here is an example of a working configuration](https://github.com/ente-io/ente/discussions/1764#discussioncomment-9478204).
1. The bucket CORS rules do not allow museum to access these objects. For
uploading files from the browser, you will need to set `allowedOrigins` to
`*`, and allow the `X-Auth-Token`, `X-Client-Package` headers configuration
too. [Here is an example of a working
configuration](https://github.com/ente-io/ente/discussions/1764#discussioncomment-9478204).
2. The credentials are not being picked up (you might be setting the correct
creds, but not in the place where museum picks them from).
credentials, but not in the place where museum reads them from).
### Mismatch in File Size
### Mismatch in file size
The "Mismatch in File Size" error mostly occurs in a situation where the client (`1`)
is re-uploading a file which is already in the bucket with a different File Size. The
reason for re-upload could be anything including Network issue, sudden killing of app
before the upload is complete and etc.
The "Mismatch in file size" error mostly occurs in a situation where the client
(`1`) is re-uploading a file which is already in the bucket with a different
file size. The reason for re-upload could be anything including network issue,
sudden killing of app before the upload is complete and etc.
This is also one of Museums (`2`) Validation Checks for the size of file being
re-uploaded from the client to the size of the file which is already
uploaded to the S3 Bucket.
In most case, it is very unlikely that this error could be a cause of some mistake in
the configuration or Browser/Bucket CORS.

30
docs/docs/self-hosting/index.md
View File

@ -10,34 +10,32 @@ the same code we use for our own cloud service.
> [!TIP]
>
> To get some context, you might find our
> [blog post](https://ente.io/blog/open-sourcing-our-server/) announcing the
> open sourcing of our server useful.
> You might find our [blog post](https://ente.io/blog/open-sourcing-our-server/)
> announcing the open sourcing of our server useful.
## System requirements
## System Requirements
The server has minimal resource requirements, running as a lightweight Go binary
with no server-side ML. It performs well on small cloud instances, old laptops,
and even [low-end embedded devices](https://github.com/ente-io/ente/discussions/594)
reported by community members. Virtually any reasonable hardware should be sufficient.
The server has minimal resource requirements, running as a lightweight Go
binary. It performs well on small cloud instances, old laptops, and even
[low-end embedded devices](https://github.com/ente-io/ente/discussions/594).
## Getting started
Execute the below one-liner command in your terminal to setup Ente on your system.
Run this command on your terminal to setup Ente.
```sh
sh -c "$(curl -fsSL https://raw.githubusercontent.com/ente-io/ente/main/server/quickstart.sh)"
```
The above `curl` command is a simple shell-script, which pulls the docker images,
creates a directory `my-ente` in the current working directory and starts all the
containers required to run Ente on your system.
The above `curl` command pulls the Docker image, creates a directory `my-ente`
in the current working directory and starts all containers required to run Ente.
![quickstart](/quickstart.png)
![self-hosted-ente](/web-app.webp)
## Queries?
If you need any help or support, do not hesitate to drop your queries on our community
[discord channel](https://discord.gg/z2YVKkycX3) or create a
[Github Discussion](https://github.com/ente-io/ente/discussions) where 100s of self-hosters help each other.
If you need support, please ask on our community
[Discord](https://ente.io/discord) or start a discussion on
[GitHub](https://github.com/ente-io/ente/discussions/).

77
docs/docs/self-hosting/museum.md
View File

@ -1,41 +1,49 @@
---
title: Building your Museum.yaml
title: Configuring your server
description: Guide to writing a museum.yaml
---
## Configuring `museum.yaml`
# Configuring your server
`Museum.yaml` is a YAML configuration file used to configure various things for museum.
By default, [`local.yaml`](https://github.com/ente-io/ente/tree/main/server/configurations/local.yaml)
is also available, but it is overridden if `museum.yaml` file is found. We highly
recommend creating and building your own `museum.yaml` instead of editing `configurations/local.yaml`.
The `my-ente` directory will include a `museum.yaml` file with some configurations around encryption
keys and secrets, postgres DB, and MinIO.
Ente's monolithic server is called **museum**.
`museum.yaml` is a YAML configuration file used to configure museum. By default,
[`local.yaml`](https://github.com/ente-io/ente/tree/main/server/configurations/local.yaml)
is provided, but its settings are overridden with those from `museum.yaml`.
If you used our quickstart script, your `my-ente` directory will include a
`museum.yaml` file with preset configurations for encryption keys, secrets,
PostgreSQL and MinIO.
> [!TIP]
> Always do `docker compose down` inside `my-ente` directory, if you've made any changes to `museum.yaml`
> and then restart the containers with `docker compose up -d ` to see the changes in action.
>
> Always do `docker compose down` inside your `my-ente` directory. If you've
> made changes to `museum.yaml`, restart the containers with `docker compose up
> -d ` to see your changes in action.
### S3 Buckets
## S3 buckets
By default, the `s3` section is configured to use local minIO buckets and for the same reason
`are_local_buckets` is set to `true`. If you wish to bring any external S3 provider,
you just have to edit the configuration with appropriate credentails and details given by the provider.
And set `are_local_buckets` to false. Check out [Configuring S3](/self-hosting/guides/configuring-s3.md)
to understand more on how to configure S3 buckets and how the communication happens.
The `s3` section within `museum.yaml` is by default configured to use local
MinIO buckets.
MinIO makes use of the port `3200` for API Endpoints and the Client Web App is run over `:3201`
(both on localhost). You can login to MinIO Console Web UI by accessing `localhost:3201` in your web-browser
and setting up all the things related to regions there itself.
If you wish to use an external S3 provider, you can edit the configuration with
your provider's credentials, and set `are_local_buckets` to `false`.
If you face any issues related to uploads then checkout
[Troubleshooting Bucket CORS](/self-hosting/troubleshooting/bucket-cors) and
[Frequently Answered Error related to S3](/self-hosting/guides/configuring-s3#fae-frequently-answered-errors)
Check out [Configuring S3](/self-hosting/guides/configuring-s3.md) to understand
more about configuring S3 buckets.
### App Endpoints
MinIO uses the port `3200` for API Endpoints and their web app runs over
`:3201`. You can login to MinIO Web Console by opening `localhost:3201` in your browser.
Ente Photos Web app is divided into multiple sub-apps like albums, cast, auth, etc.
These endpoints are configurable in the museum.yaml under the `apps.*` section.
If you face any issues related to uploads then checkout [Troubleshooting bucket
CORS](/self-hosting/troubleshooting/bucket-cors) and [Frequently encountered S3
errors](/self-hosting/guides/configuring-s3#frequently-encountered-errors).
## App endpoints
Ente Photos Web app is divided into multiple sub-apps like albums, cast, auth,
etc. These endpoints are configurable in the museum.yaml under the `apps.*`
section.
For example,
@ -47,14 +55,15 @@ apps:
family: family.myente.xyz
```
By default, all the values redirect to our publicly hosted production services.
After you are done with filling the values, restart museum and the App will start utilizing
those endpoints for everything instead of the Ente's prod instances.
By default, all the values redirect to our publicly hosted production services.
After you are done with filling the values, restart museum and the app will
start utilizing those endpoints instead of Ente's production instances.
Once you configure all the necessary endpoints, `cd` into `my-ente` and stop all the docker
containers with `docker compose down` to completely stop all the containers and restart them
with `docker compose up -d`.
Once you have configured all the necessary endpoints, `cd` into `my-ente` and
stop all the Docker containers with `docker compose down` and restart them with
`docker compose up -d`.
Similarly, you can read the default [`local.yaml`](https://github.com/ente-io/ente/tree/main/server/configurations/local.yaml)
and build a functioning `museum.yaml` for many other functionalities like SMTP, Discord
Notifications, Hardcoded-OTT's, etc.
Similarly, you can use the default
[`local.yaml`](https://github.com/ente-io/ente/tree/main/server/configurations/local.yaml)
as a reference for building a functioning `museum.yaml` for many other
functionalities like SMTP, Discord notifications, Hardcoded-OTTs, etc.

39
docs/docs/self-hosting/reverse-proxy.md
View File

@ -3,35 +3,34 @@ Title: Configuring Reverse Proxy
Description: configuring reverse proxy for Museum and other endpoints
---
## Reverse Proxy
# Reverse proxy
This step isn't really the direct next step after creating an account. It is
one of the most essential steps to avoid certain CORS errors and will help you through
the configuration coming ahead.
Ente's server (museum) runs on port `:8080`, web app on `:3000` and the other
apps from ports `3001-3004`.
Museum runs on port `:8080`, Ente Photos web app runs on `:3000` and so on the other apps
are lined up after each other from ports `3001-3004`.
We highly recommend using HTTPS for Museum (`8080`). For security reasons museum
will not accept incoming HTTP traffic.
We highly recommend using HTTPS for Museum (`8080`). Primarily, because for security reasons Museum
won't accept any incoming HTTP traffic. Hence, all the requests will fail.
Head over to your DNS Management Dashboard and setup the appropriate records for the endpoints.
Mostly, `A` or `AAAA` records targeting towards your server's IP address should be sufficient. The rest of the work
will be done by the web server sitting on your server machine.
Head over to your DNS management dashboard and setup the appropriate records for
the endpoints. Mostly, `A` or `AAAA` records targeting towards your server's IP
address should be sufficient. The rest of the work will be done by the web
server on your machine.
![cloudflare](/cloudflare.png)
### With Caddy
### Caddy
Setting up a reverse proxy with Caddy is pretty easy and straightforward. Firstly, install Caddy
on your server machine.
Setting up a reverse proxy with Caddy is easy and straightforward.
Firstly, install Caddy on your server.
```sh
sudo apt install caddy
```
After the installation is complete, a `Caddyfile` is created on the path `/etc/caddy/`. This file is
used to configure reverse proxies and a whole lot of different things.
After the installation is complete, a `Caddyfile` is created on the path
`/etc/caddy/`. This file is used to configure reverse proxies among other
things.
```yaml
# Caddyfile - myente.xyz is just an example.
@ -44,5 +43,7 @@ ente.myente.xyz {
#...and so on for other endpoints
```
After a few hard-reloads, Ente Photos web app should be up on https://ente.myente.xyz. You can check out
the documentation for any other reverse proxy tool (like nginx) you want to use.
After a hard-reload, the Ente Photos web app should be up on https://ente.myente.xyz.
If you are using a different tool for reverse proxy (like nginx), please check
out their documentation.