What is the relationship really?

Let's compare a bit, it may be funny, but let's do it.

what are a real relationship between People?

• Right relationships are lasting and long -term.
• Both parties accept responsibility for relationships and changes.
• Changes on one side of the relationship also affect the other side.
• The two sides of a relationship live together.
• If the relationship leads to the birth of a child, both parties will accept the relationship.
• If one party dies, especially if it's a lover, the other party probably won't want to live either.

Now let's look at the relationship features in SQL:

• There is no real relationship. The two sides have only one connection.
• Relationships are not together. And each lives independently.
• Relationships are not deep.
• Relationships do not give birth to any children. (In Lesan, you will see that relationships encourage you to create new models)
• If we want to delete one side of the relationship, especially if the other side is dependent on this side, we will only receive an error message.
• And the most important thing is that it is not clear what kind of effects each relationship we create will have on the other side of the relationship.

What are the relationships in NoSQL?

• There is no real relationship. In fact, there is no proper connection between the two sides.
• If we consider embeding as a relationship:
  • the changes of each party have no effect on the other side and cause many inconsistencies in the data.
  • the two sides leave each other after the relationship. Infact, it is not clear what kind of effects each relationship we create will have on the other side of the relationship.
• In this type of databases, they prevent the child from being born, and if a child is born, only one side will be informed of it and probably will not take much responsibility for it.
• There is no management in deleting information and they are easily deleted by either side of the relationship.

And finally what are the relationships in Lesan:

• Relationships are as strong as possible, and are described in detail when creating a model.
• Relationships fully contain each other's pure properties within themselves.
• If a relationship changes, all related parties will be notified and apply the changes according to a process.
• By establishing a relationship and seeing many changes on one side of this relationship, you are encouraged to create new relationships. Don't worry, this issue will not add more complexity to the data model, but it will also make the data more understandable. (Below there is an example to understand this)
• Having complete information about relationships, we can prevent the deletion of a document that other documents are dependent on with an error message, and we can recursively delete all dependent documents by setting a small option.
• And the most important point is that it is exactly clear what effects each relationship we have created will have on the other side of the relationship.

Example

SQL

So let's go back to our example (countries, cities and users)
If we want to define a relationship between the country, city and user models in SQL, this relationship will be as follows (The code below is written for PostgreSQL):

CREATE TABLE country (
  id serial PRIMARY KEY,
  name VARCHAR ( 50 ) UNIQUE NOT NULL,
  abb VARCHAR ( 50 ) NOT NULL,
  population INT NOT NULL,
);

CREATE TABLE city (
  id serial PRIMARY KEY,
  name VARCHAR ( 50 ) UNIQUE NOT NULL,
  abb VARCHAR ( 50 ) NOT NULL,
  population INT NOT NULL,
  country_id INT NOT NULL,
  FOREIGN KEY (country_id)
    REFERENCES country (country_id),
);

CREATE TABLE user (
  id serial PRIMARY KEY,
  name VARCHAR ( 50 ) UNIQUE NOT NULL,
  age INT NOT NULL,
  country_id INT NOT NULL,
  FOREIGN KEY (country_id)
    REFERENCES country (country_id),
  city_id INT NOT NULL,
  FOREIGN KEY (city_id)
    REFERENCES city_id (city_id),
);

Pay attention that the relationships are separated from each other as much as possible and only one ID is kept on one side. Whenever we need to know the details of the relationship, we have to visit both sides of the relationship.
For example, let's imagine that we want the cities of Iran, we must first find Iran and then filter the city using Iran ID.
Now let's imagine that we want to find the country of Iran along with its 50 most populated cities, we have to find Iran first, then find the cities according to the ID filter of the country of Iran along with the sort based on the city population field and the limit of 50.
Let's run a more complex query. Suppose we want to receive 50 most populous cities from the 50 most populous countries in the world.
Or we want to find the oldest people in the 50 most populous countries in the world.
To get the above cities or users, we have to create and execute much more complex queries that may be time-consuming in some cases, although there are alternative ways such as creating separate tables in SQL for these specific purposes, but these ways also add a lot of complexity to the project.

NoSQL

What if we could do the above with just a simple query? NoSQL is designed for this, let's see how these tables are implemented in NoSQL databases (Here we have used mongoose so that we can have the shape of the schemas):

const CountrySchema = new mongoose.Schema ({
	name: String,
	abb: String,
	population: Number,
});
const Country = mongoose.model("Country", CountrySchema)

const CitySchema = new mongoose.Schema ({
	name: String,
	abb: String,
	population: Number,
	country: {
		type: mongoose.Schema.Types.ObjectId,
		ref: Country
	} 
});
const City = mongoose.model("City", CitySchema)

const UserSchema = new mongoose.Schema ({
	name: String,
	age: Number,
	country: {
		type: mongoose.Schema.Types.ObjectId,
		ref: Country
	},
	city: {
		type: mongoose.Schema.Types.ObjectId,
		ref: City
	} 
});
const User = mongoose.model("User", CitySchema)

The code above is exactly equivalent to the code we wrote for PostgreSQL and creates exactly the same tables in MongoDB. All the issues we described for SQL will be present here as well, but wait, we can add other fields to these tables to simplify the complex queries we talked about above.

We can store its cities inside each country by adding a field called cities. Pay attention to the following code:

const CountrySchema = new mongoose.Schema ({
  name: String,
  abb: String,
  population: Number,
  cities: [{
    name: String,
    abb: String,
    population: Number,
  }]
});

Now we can get a country along with its cities just by sending a single query. For example, we can get the country of Iran along with its cities with just one query from the database. But wait, some new issues have arisen.

• How should we save the cities inside the country ?
  For this, it is necessary to find the country associated with the city in the function we write to add the city and add this new city to the cities field of that country. This means that when adding a city in the table of cities, we must insert a new record and edit a record in the table of countries.
• Can we store all the cities of a country within itself ?
  The short answer is no, although it is possible that the number of cities in a country can be stored within the country, but in some situations, the number of documents that we need to store inside another document may be very large, such as the users of a country. So what should we do? Save a limited number of cities, how many? The number that we feel should be requested in the first pagination (for example, 50 numbers). So, in the function we have written to store the city, we must be aware that if the field of cities within the country has stored 50 cities within itself, do not add this new city to this field.
• What if a city changes ?
  Well, as a rule, we should find the country related to that city and check whether this city is stored in the field of cities of that country or not, if yes, we should correct it.
• What if a city is removed ?
  We need to find the country associated with the city and if this city is present in the field of cities of that country, we should modify that field as well. How? First, we remove this city from the array of cities, then we check whether this field has reached the limited number that we have previously considered for it, if yes, then this country may have other cities that are not in this field. So we need to find one of them and add it to this field.

All this was added just so that we can have its cities when receiving a country!
Don't worry, it's worth it. Because we usually add cities and countries once, besides, its information does not change much (except for the population, which we will talk about later). And on the other hand, these cities and countries will be received many times.

Now, what if we want to get the 50 most populous countries along with the 50 most populous cities of that country?
We can add a new field to the country. Pay attention to the following code:

const CountrySchema = new mongoose.Schema ({
  name: String,
  abb: String,
  population: Number,
  cities: [{
    name: String,
    abb: String,
    population: Number,
  }],
  mostPopulousCities: [{
    name: String,
    abb: String,
    population: Number,
  }],
});

For mostPopulousCities Field, we should consider all the events that happened above, although with a slight change:

• What if a city changes ? We need to find the country associated with the city, then see if this city is stored in the cities and mostPopulousCities fields of this country or not. If it is stored in the cities field, we must do the same steps as above, but if it is stored in the mostPopulousCities field, we must first, see which city field has changed, if the population field has changed, this city may no longer be included in this list and we want to remove it from the list and add another city to this list based on its population, otherwise it is possible This city is not in the mostPopulousCities list at all, but due to the change in the city's population, we want to add it to this list. Note that this city may be added anywhere in this list, and on the other hand, if this list has reached the end of the predetermined capacity, we must remove a city from the end.

Let's look at this issue from the other side of the relationship.
What if we want to access the country related to that city from within the cities?
Well, for this purpose, we can add a field called the country in each city. And instead of storing only the country's ID in it, embed all the country's information in it.

const CitySchema = new mongoose.Schema ({
  name: String,
  abb: String,
  population: Number,
  country: {
    name: String,
    abb: String,
    population: Number,
  } 
});

The good thing here is that this field is no longer an array, it's just an object, so we don't have any of the calculations we had to manage cities in the country. And it is enough to find the country related to the city in the function we wrote to add the cities and put it as the value of the country field in the city.
But the critical issue here is that if the country is updated, we must find all the cities related to that country and update the country stored inside the cities as well. Sometimes this number may be very high, for example, consider the country of China or India and put users instead of the city and imagine that all the people of this country are registered in this software, in this case with every update The country should update at least another billion documents (there are different solutions for this problem in the Lesan, which you will see below). Finally, our mongoose model will probably look like this:

  const CountrySchema = new mongoose.Schema ({
  name: String,
  abb: String,
  population: Number,
  cities: [{
    name: String,
    abb: String,
    population: Number,
  }],
  mostPopulousCities: [{
    name: String,
    abb: String,
    population: Number,
  }],
});
const Country = mongoose.model("country", CitySchema);

const CitySchema = new mongoose.Schema ({
  name: String,
  abb: String,
  population: Number,
  country: {
    name: String,
    abb: String,
    population: Number,
  } 
});

const City = mongoose.model("City", CitySchema)

const UserSchema = new mongoose.Schema ({
	name: String,
	age: Number,
	country: {
		type: mongoose.Schema.Types.ObjectId,
		ref: Country
	},
	city: {
		type: mongoose.Schema.Types.ObjectId,
		ref: City
	} 
});
const User = mongoose.model("User", CitySchema)

Lesan

So, if we want to create the same relationships with ‌Lesan, what should we do? Just enter the code below:

// Country Model
const countryCityPure = {
  name: string(),
  population: number(),
  abb: string(),
};

const countryRelations = {};

const countries = coreApp.odm.newModel(
  "country",
  countryCityPure,
  countryRelations,
);

// City Model
const cityRelations = {
  country: {
    optional: false,
    schemaName: "country",
    type: "single" as RelationDataType,
    relatedRelations: {
      cities: {
        type: "multiple" as RelationDataType,
        limit: 50,
        sort: {
          field: "_id",
          order: "desc" as RelationSortOrderType,
        },
      },
      mostPopulousCities: {
        type: "multiple" as RelationDataType,
        limit: 50,
        sort: {
          field: "population",
          order: "desc" as RelationSortOrderType,
        },
      },
    },
  },
};

const cities = coreApp.odm.newModel(
  "city",
  countryCityPure,
  cityRelations,
);

// User Model
const userPure = {
  name: string(),
  age: number(),
};

const userRelations = {
  country: {
    optional: false,
    schemaName: "country",
    type: "single" as RelationDataType,
    relatedRelations: {
      users: {
        type: "multiple" as RelationDataType,
        limit: 50,
        sort: {
          field: "_id",
          order: "desc" as RelationSortOrderType,
        },
      },
    },
  },
  city: {
    optional: false,
    schemaName: "country",
    type: "single" as RelationDataType,
    relatedRelations: {
      users: {
        type: "multiple" as RelationDataType,
        limit: 50,
        sort: {
          field: "_id",
          order: "desc" as RelationSortOrderType,
        },
      },
    },
  },
};

const users = coreApp.odm.newModel(
  "user",
  userPure,
  userRelations,
);

In the code above, we have not defined any relationship for the country, but in fact, the country is related to both the city and the user, but this relationship is defined by them because they were the requesters of the relationship.
If you pay attention, we have defined two relatedRelations for the country when defining city relations, which causes two fields called cities and mostPopulousCities to be added to the country schema. For the cities field, we have set the sort on _id and in descending order, and we have limited the capacity of the field to 50 numbers with the limit option, which causes the last 50 cities of each country to be stored in it.
But in the mostPopulousCities field we have once again stored 50 cities in each country, but this time by sorting on the City population field.
The important thing here is that all the things we said we need to do in NoSQL databases using Mongoose are done automatically in Lesan and you don't need any additional code to manage these relationships during insert, update or delete. All work will be done by Lesan.

Test Realation in Lesan

Clone and run E2E

you can clone lesan repo by git clone https://github.com/MiaadTeam/lesan.git command and the go to tests/playground folder and run e2e.ts file by execute this command: deno run -A e2e.ts you should see this output:

HTTP webserver running.
please send a post request to http://localhost:1366/lesan
you can visit playground on http://localhost:1366/playground

Listening on http://localhost:1366/

Visit Playground

Now you can visit the playground at http://localhost:1366/playground and send requests to the server for country, city, and user models:

and use addCountry, addCountries, updateCountry, getCountries, deleteCountry methods for country models:

also use addCity, updateCity, addCities, getCities and addCityCountry for city model:

and also use addUser, addUsers, addUserLivedCities, addUserCountry, addUserCities, addMostLovedCity, removeMostLovedCity, removeLivedCities, updateUser, getUser and getUsers for user model:

you can find e2e.ts raw file here and see all functions write in it.

Visit Schema and Act

You can see all schema information including pure, mainRelation and relatedRelation inside schema modal box at playground when clicking on Schema button:

here is the screenshot of schema modal box:

Also you can see all Act information including service, model, act and its inputs such as set and get inside act modal box at playground when clicking on Act button:

here is the screenshot of act modal box:

Visit E2E test modal

We have already prepared several E2E test series for e2e.ts's file, and you can go to E2E modal box by clicking the E2E test button:

here you can import E2E test config file by clicking on import button:

We have these 3 json files next to the e2e.ts file, all three of which can be used for E2E testing:

• Configdata.json
• fakerTest.json
• stress.json

Configdata E2E file

In the config.json file, all the functions written in e2e.ts have been tested. In fact, all the important functions, including all the functions of the ODM section in Lesan, have been tested in this file. Let us see all the parts of this E2E test one by one (The point is that in almost all the functions written in ODM, relationships are important and Lesan must manage them.):

1. create new country with main → country → addCountry:
   

   1. here we used main service and country model and addCountry act.
   2. we repeat this section 15 times.
   3. we captured countryId from last response of this sections.
   4. here we used faker for create name, population and abb for new country.

2. create multiple country with one request with main → country → addCountries:

   1. here we used main service and country model and addCountries act.
   2. here we insert an array of country for multiCountries key inside set object.
   3. we captured some country ID from request response.

3. create new city with main → city → addCity:

   1. here we used iranId the captured variable we get from request number 2 response.
   2. we captured body._id with the haratId from response.

4. create multiple city with main → city → addCities:

   1. here we insert an array of city for multiCities key inside set object.
   2. here we used iraqId the captured variable we get from request number 2 response.

5. create multiple city with main → city → addCities:

   1. here we insert an array of city for multiCities key inside set object. We also used faker here.
   2. here we used afghanId the captured variable we get from request number 2 response.

6. change country relation of a city with main → city → addCityCountry:

   1. here we used haratId and afghanId captured variables. please check mongodb compass beacuase the both side of relation are changend.

7. create new city with main → city → addCity:

8. create new city with main → city → addCity:

   1. please note that we set isCapital field to true so the capital field of related country is filled with this city.

9. create new city with main → city → addCity:

10. create new city with main → city → addCity:

11. create new city with main → city → addCity:

12. create new city with main → city → addCity:

13. create new city with main → city → addCity:

14. create new city with main → city → addCity:

15. create new city with main → city → addCity:

16. create new city with main → city → addCity:

17. create new city with main → city → addCity:

18. create new city with main → city → addCity:

19. just get list of countries with main → country → getCountries:

20. create new city with main → user → addUser:

    1. the country were user lived is Iran.
    2. the user lived in two city: Hamedan and Tehran.

21. create new city with main → user → addUser:

22. create new city with main → user → addUser:

23. create new city with main → user → addUser:

24. create new city with main → user → addUser:

25. create new city with main → user → addUser:

26. create new city with main → user → addUser:

27. create new city with main → user → addUser:

28. create new city with main → user → addUser:

29. change country relation of a user with main → user → addUserCountry: 1.we just send a country ID with a user ID and with a simple function all magic happen in both side of relation.

30. change country relation of a user with main → user → addUserCountry:

31. add city to livedCities relation of a user with main → user → addUserLivedCities: 1.we just send list of city ID with a user ID and with a simple function all magic happen in both side of relation.

32. add city to livedCities relation of a user with main → user → addUserLivedCities:

33. remove city from livedCities relation of a user with main → user → removeLivedCities: 1.we just send list of city ID with a user ID and with a simple function all magic happen in both side of relation.

34. add a city to mostLovedCity relation of a user with main → user → addMostLovedCity:

35. add a city to mostLovedCity relation of a user with main → user → addMostLovedCity:

36. remove a city from mostLovedCity relation of a user with main → user → removeLivedCities:

37. update a country with main → country → updateCountry:

    1. We send the ID of a country along with the rest of the pure fields (pure fields are optional) to it, and in a simple function the following will happen:
       • Update the country itself
       • Updating the country field in all cities related to that country
       • Updating the country field in all users related to that country

    The point to be mentioned here is that you should not send another field for updating other than pure fields. Because the relationships must be completely managed by Lesan himself.

38. update a city with main → city → updateCity:

39. update a user with main → user → updateUser:

After clicking the run E2E test button, you will go to the test results page.

If you scroll down a little, you can see the results of each sequence separately:

1. with this button you can change view of panel from body-header & Description to REQUEST & RESULT
2. show some description about sequence including request number & timing, captured value and so on.
3. show unparsed header and body you send to the backend.
4. show the index of each sequence.
5. show response get back from server.
6. show parsed request you send to server, including parsed header and body.
7. pagination for sequence with more than 1 request.

After finished executing all test in configdata.json you have a nice data inserted to sample collection in mongodb.

You can play with this data in playground and change everything you want.

fakerTest E2E file

This file is not very important in this section, it is only used to test faker functions in E2E.

stress E2E file

This file is used to test the insertMany that has a relation with it. Note that a very large number of server-side requests are sent, resulting in the creation of a country with 50,000 cities for that country and 50,000 users for that country.

1. create a country with main → country → addCountry:

2. create 50,000 cities with main → city → addCities:

3. create 50,000 users with main → user → addUsers:

After clicking the run E2E test button, 10001 requests should be sent to the server, and as a result, a country, 50,000 cities, and 50,000 users should be created.

Pay attention to the entered data, although we have used insertMany, all relationships are embedded.
In the country, we have embedded the cities in 4 fields separately and with different conditions. And we have embedded users in two fields with different conditions.

In the cities, we have embedded the respective country.

In the user schema, for each user, we have embedded the cities he has lived in as a list, the city he is most interested in as an object, and the country of each user as an object.

The interesting thing about this E2E test is that after the database is filled, you can test a big update in Playground. If you update the country in Playground, 100,000 other documents must be updated along with the country record itself.

before execute main → country → updateCountry: executin main → country → updateCountry: after execute main → country → updateCountry:

relationship sweets in Lesan

shoma tanha ba fieldhaye pure yek schema sar o kar darid va modiriat rabeteha tamaman be sorat khodkar tavasot lesan anjam mishavad. shoma mitavanid bar asas rabeteye yek schema an ra sort ya filter konid shoma baraye daryaft dadaha ba queryhaye pichide asnad besiyar kamtari ra az database jamavari mikonid. (link bedam be tozihat kamel)

relationship bitterness in Lesan

barkhi az rabeteha baes eijad updatehaye besiyar bozorg mishavand. rah hal: 1-eijad rabeteye jadid 2-qq 3-in-memory db

اول راجع به ایمکه رابطه چی هست حرف می‌زنم، بعد می‌گم اس‌کیوال فقط کانکشن برقرار می‌کنه، بعد می‌گم نواس‌کیوال هم فقط امبد می‌کنه و مدیریت درست نداره.

بعد میام راجع به اینکه هر فیلد پر تغییری می‌تونه به رابطه تبدیل بشه حرف می‌زنم، مثال بانک و ثبت احوال کشورها رو می‌گم.

بعد میام راجع به اینکه رابطه‌های دو سر چندتایی نمی‌تونه دو سر بی انتها داشته باشه حرف می‌زنم و چندتا مثال می‌زنم.

بعد راجع به آپدیت شدن رابطه‌ها حرف می‌زنم.

بعد راجع به دیتابیس اینمموری حرف می‌زنم.

بعد راجع به مدیریت کیوکیو حرف می‌زنم.

حتما یادم باشه راجع به اینکه وقتی امبد می‌کنی چقدر دریافت داده‌ها راحت هست هم حرف بزنیم، از اون طرف راجع به اینکه فیلتر کردنشون بر اساس فیلد امبد شده چه معجزه‌ای میکنه هم حرف بزنم

حتما راجع به اینکه چه نکته‌هایی داره طراحی مدل توی لسان حرف بزنم یعنی اینکه بگم رابطه‌ها از یک طرف تعریف می‌شن بعد توی طرف بعدی از همینجا ساید افکت‌هاش مشخص میشه بگم و اینکه نگران نباشن چون توی پلی‌گراند می‌تونن برای یک مدل همه‌ی رابطه‌هایی که داره چه از طرف خودش تعریف شده باشه چه بقیه براش تعریف کرده باشن رو ببینه و یه عکس از پلی‌گراند بذارم، نکته مهمش اینه که بگم همیشه رابطه رو از طرف مهمش درخواست بدن تا بشه هر چقدر می‌خوایم ساید افکت مناسب براش بذاریم

این رو هم بنویسم که در واقع مشکلات آپدیت و دیلیت و اینزرت همین الآن هم هم توی لایه ی کش سرور هم توی سی دی ان ها وجود داره و ما این معضلات رو آوردیم توی لاجیک بک اند و ساده سازی کردیم و در واقع کلی هم از اتلاف انرژی و چیزای دیگه هم اینجا جلوگیری کردیم علاوه بر این اینکه اکثر این عملیات ها رو هم خودکار کردیم.