Commit Graph

19 Commits

Author SHA1 Message Date
Corey
16b1b2a197 feat: support relativeTime query constraint on Postgres (#7747) 2022-01-02 01:10:54 +01:00
Diamond Lewis
a02014f557 Improve single schema cache (#7214)
* Initial Commit

* fix flaky test

* temporary set ci timeout

* turn off ci check

* fix postgres tests

* fix tests

* node flaky test

* remove improvements

* Update SchemaPerformance.spec.js

* fix tests

* revert ci

* Create Singleton Object

* properly clear cache testing

* Cleanup

* remove fit

* try PushController.spec

* try push test rewrite

* try push enqueue time

* Increase test timeout

* remove pg server creation test

* xit push tests

* more xit

* remove skipped tests

* Fix conflicts

* reduce ci timeout

* fix push tests

* Revert "fix push tests"

This reverts commit 05aba62f1cbbca7d5d3e80b9444529f59407cb56.

* improve initialization

* fix flaky tests

* xit flaky test

* Update CHANGELOG.md

* enable debug logs

* Update LogsRouter.spec.js

* create initial indexes in series

* lint

* horizontal scaling documentation

* Update Changelog

* change horizontalScaling db option

* Add enableSchemaHooks option

* move enableSchemaHooks to databaseOptions
2021-03-16 16:05:36 -05:00
Diamond Lewis
9563793303 Allow single server instance in test suite (#7262)
* initial pass

* reconfigureServer when needed

* finish postgres tests

* mongo tests

* more tests

* clean up

* re-add skipped test

* Fix transaction tests

* handle batch

* AuthenticationAdapter fix

* More reconfiguration

* clean up

* properly terminate cli servers

* handle Parse.Push

* Flaky PushController

* ensure reconfigureServer when changed

* fix postgres tests

* remove console.log

* LiveQuery spec remove duplicates and listeners
2021-03-13 09:05:22 -06:00
Diamond Lewis
39fa17f914 Postgres: Properly initialize database on startup and debugger (#7255)
* PG: Properly initialize database

* fix flaky tests

* flaky test

* correct test

* no idea

* clean up debugger
2021-03-10 11:31:35 -08:00
Corey
2b9b336dd9 Fix flaky postgres test (#7228)
* Fix flaky postgres test

* add pgcrypto extension to test

* fix postgres ref link CONTRIBUTING guide and add pgcrypto extension

* Update CONTRIBUTING.md
2021-02-25 17:28:15 -06:00
Diamond Lewis
e6ac3b6932 fix(prettier): Properly handle lint-stage files (#6970)
Now handles top level files and recursive files in folders.

Set max line length to be 100
2020-10-25 15:06:58 -05:00
Corey
de7ec58de3 Postgres: prepend className to unique indexes (#6741)
* prepend className to unique index to allow multiple unique indexes for different classes

* add testcase

* switched test so it can be tested on older versions of parse-server and show failure

* get rid of console log messages on restart by checking if the index exists before creating it

* add IF NOT EXISTS and IF EXISTS to ALTER TABLE

* revert some of code

* ensureIndex use IF NOT EXISTS

* ALTER TABLE CONSTRAINT can't use IF, ADD/DROP COLUMN can

* retesting

* update

* switchted to CREATE UNIQUE INDEX instrad of ALTER TABLE... ALTER TABLE doesn't seem to be needed
2020-10-11 22:47:45 -07:00
Corey
dcba8e0b7c Case insensitive username and email indexing and query planning for Postgres (#6506)
* Update .travis.yml

testing error to see what happens...

* Update .travis.yml

Attempting to resolve postgres in CL by installing postgis via sudo instead of through apt/packages

* Update .travis.yml

* Update .travis.yml

* Update .travis.yml

Removed extra lines of postgres that were under "services" and "addons". I believe the "postgresql" line under "services" was installing the default of 9.6 and "addons" was installing postgres 11. My guess is the fail was occurring due to 9.6 being called sometimes and it never had postgis installed. If this is true, the solution is to only install one version of postgres, which is version 11 with postgis 2.5.

* Adding test case for caseInsensitive 

Adding test case for verifying indexing for caseInsensitive

* Implementing ensureIndex

* Updated PostgresStorageAdapter calls to ST_DistanceSphere. Note this has a minimum requirement of postgis 2.2. Documented the change in the readme. This is address #6441

* updated postgres sections of contributions with newer postgres info. Also switched postgis image it points to as the other one hasn't been updated in over a year.

* more info about postgres

* added necessary password for postgres docker

* updated wording in contributions

* removed reference to MacJr environment var when starting postgres in contributions. The official image automatically creates a user named 'postgres', but it does require a password, which the command sets to 'postgres'

* added more time to docker sleep/wait to enter postgis commands. This will always take a few seconds because the db is installing from scratch everytime. If postgres/postgis images aren't already downloaded locally, it will take even longer. Worst case, if the command times out on first run. Stop and remove the parse-postgres container and run the command again, 20 seconds should be enough wait time then

* latest changes

* initial fix, need to test

* fixed lint

* Adding test case for caseInsensitive 

Adding test case for verifying indexing for caseInsensitive

* Implementing ensureIndex

* Updated PostgresStorageAdapter calls to ST_DistanceSphere. Note this has a minimum requirement of postgis 2.2. Documented the change in the readme. This is address #6441

* updated postgres sections of contributions with newer postgres info. Also switched postgis image it points to as the other one hasn't been updated in over a year.

* more info about postgres

* added necessary password for postgres docker

* updated wording in contributions

* removed reference to MacJr environment var when starting postgres in contributions. The official image automatically creates a user named 'postgres', but it does require a password, which the command sets to 'postgres'

* added more time to docker sleep/wait to enter postgis commands. This will always take a few seconds because the db is installing from scratch everytime. If postgres/postgis images aren't already downloaded locally, it will take even longer. Worst case, if the command times out on first run. Stop and remove the parse-postgres container and run the command again, 20 seconds should be enough wait time then

* latest changes

* initial fix, need to test

* fixed lint

* Adds caseInsensitive constraints to database, but doesn't pass regular tests. I believe this is because ensureIndex in the Postgres adapter is returning wrong. Also, some issues with the caseInsensitive test case

* this version addes the indexes, but something still wrong with the ensureIndex method in adapter

* removed code from suggestions

* fixed lint

* fixed PostgresAdapter test case

* small bug in test case

* reverted back to main branch package.json and lock file

* fixed docker command in Contribute file

* added ability to explain the find method

* triggering another build

* added ability to choose to 'analyze' a query which actually executes (this can be bad when looking at a query plan for Insert, Delete, etc.) the query or to just setup the query plan (default, previous versions defaulted to 'analyze'). Alse added some comparsons on sequential vs index searches for postgres

* made sure to check that search actually returns 1 result. Removed prep time comparison between searches as this seemed to be variable

* added test cases using find and case insensitivity on fields other than username and password. Also added explain to aggregate method

* fixing issue where query in aggregate replaced the map method incorrectly

* reverted back to mapping for aggregate method to make sure it's the issue

* switched back to caseInsensitive check for email and username as it was causing issues

* fixed aggregate method using explain

* made query plain results more flexible/reusable. Got rid of droptables as 'beforeEach' already handles this

* updated CONTRIBUTING doc to use netrecon as default username for postgres (similar to old style). Note that the official postgres docker image for postgres requires POSTGRES_PASSWORD to be set in order to use the image

* left postgis at 2.5 in the contributing document as this is the last version to be backwards compatibile with older versions of parse server

* updating docker command for postgres

Co-authored-by: Arthur Cinader <700572+acinader@users.noreply.github.com>
2020-04-03 09:24:56 -05:00
Corey
7121ea0a16 Fixed issue that prevented Postgres Tests from passing locally and on any port other than 5432 in travis (#6531)
* Attempting to fix Postgres issue

* Attempting to fix Postgres issue

trying to stop loop

* Attempting to fix Postgres

isolating postgres calls

* Attempting to fix Postgres issue

Separating jobs

* Attempting to fix postgres

* Attempting to fix postgres

* Attempting to fix postgres

Separating builds again

* Attempting to fix postgres

* Attempting to fix postgres

* Attempting to fix postgres

Just added back version 10, just in case it gets called

* Attempting to fix postgres

* Attempting to fix postgres

* Attempting to fix postgres

* Attempting to fix postgres

* Attempting to fix postgres

* Attempting to fix postgres

* Attempting to fix postgres

* Attempting to fix postgres

* Attempting to fix postgres

* Attempting to fix postgres

* Update .travis.yml

* Attempting to fix postgres

Removed postgres installs from unneeded test cases. Added the ability to test Postgres 10 and 11

* Attempting to fix postgres

* Attempting to fix postgres

* Attempting to fix postgres

* Attempting to fix postgres

Added test for postgres 12 that's allowed to fail

* Attempting to fix postgres

* Attempting to fix postgres 

Second round to see if it fails eventually

* Attempting to fix postgres

Round 3

* Attempting to fix postgres

Allowing all postgres to fail since it seems to occur randomly

* Temporary fix: separated mongo and postgres in travis

Now the mongo and postgres scripts are independent of each other to prevent the `ERROR:  could not access file "$libdir/postgis-2.4": No such file or directory` of showing up in the rest of the builds.

In addition, a test for postgres-12 has been added for future compatibility. Both the postgres-11 and postgres-12 have been added to `allow_failures` because the aforementioned error still creeps up. Important note is that the error has nothing to do with compatibility with postgres, but rather seems to be an error of how postgres (or really postgis) is being referenced in the respective travis distribution. Lastly, this error, if truly random should appear less than before as the postgres scripts aren't being run for every build as it previously was running.

* Allowing all postgres to fail

* Allowing multiple names to fail

* Removing preinstalled versions of postgres from list

Seeing if this gets rid of the random error

* Use postgres made for dist

* Second round

* Round 3

* Round 4

* Round 5

* Fixed issue with random postgres fail

Removing the native postgres builds at the right time seems to have fixed the random error from before.

The postgres tests are now not allowed to fail.

* Added back postgres 11 and 12 to allow_failures

The actual problem is fixed, but it seems there are some instability with some of the test cases for postgres that need to be addressed at another time.

The issues that pop up are: 
- Postgres-11
```Failures:
1) Cloud Code cloud jobs should set the message / success on the job
  Message:
    Expected undefined to equal 'hello'.
  Stack:
    Error: Expected undefined to equal 'hello'.
        at <Jasmine>
        at req.message.then.then.jobStatus (/home/travis/build/parse-community/parse-server/spec/CloudCode.spec.js:1571:46)
        at process._tickCallback (internal/process/next_tick.js:68:7)
```

- Postgres-12
``` 
Failures:
1) Cloud Code cloud jobs should set the message / success on the job
  Message:
    Expected undefined to equal 'hello'.
  Stack:
    Error: Expected undefined to equal 'hello'.
        at <Jasmine>
        at req.message.then.then.jobStatus (/home/travis/build/parse-community/parse-server/spec/CloudCode.spec.js:1571:46)
        at process._tickCallback (internal/process/next_tick.js:68:7)
  Message:
    Expected 'running' to equal 'succeeded'.
  Stack:
    Error: Expected 'running' to equal 'succeeded'.
        at <Jasmine>
        at promise.then.then.jobStatus (/home/travis/build/parse-community/parse-server/spec/CloudCode.spec.js:1580:45)
        at process._tickCallback (internal/process/next_tick.js:68:7)
```

* added travis scripts for postgres

* Setting up before_install and before_script

This should shrink the footprint of the file and and reduce the redundancy of calls for postgres.

Added support for testing of Postgres 9 and 10 in the scripts, not adding the tests though

* make scripts executable

* Update .travis.yml

* add sourcing in script

* trying to fix source

* fixing env var in script

* fixed ; near then

* Cleaning up travis file

removed old lines

* Finishing clean up

* Fixing allow_failures since "name" was removed

* Update .travis.yml

* Removed Postgres 11 from allow_failures

* I think using travis default postgres port of 5433 will allow us to not have to remove anything from the image

* Switching travis to postgres port 5433

* modifying script for test

* modifying script for test

* modifying script for test

* reverting back to working way with removing postgres from image

* Reverted back to removing postgres from image

* removing postgres 12

* removed postgres-12 from allow_failures

* updated postgres method from deprecated. Also updating postgis to 3.0

* updated postgis to 3.0

* Update .travis.yml

* fix Postgres test issues that causes some tests to fail locally when using PARSE_SERVER_TEST_DATABASE_URI

* removed dependence on creating adapter locally in testcase and use config instead.

* fixed one more potential issue with postgres adapter

* modified travis and script to leverage PARSE_SERVER_TEST_DATABASE_URI

* changes to before install to reduce time

* add updated port to script for psql connection

* fixed arg for psql

* added back conf file as it probably contains the default permissions

* seeing if stopping/starting postgres back-to-back was the issue

* checking fix

* revert to working version of install script

* leave default port

* removing uninstall of postgres from script

* looks like postgres won't start until the others are removed

* forgoing using posgres service and starting directly, hoping it allows to skip remove

* removed sudo for starting postgres

* adding back sude, but using default user for postgres

* fix

* don't start service

* init db cluster before starting

* remove folder befor init

* remove folder befor init

* still have to stop service

* switch ports after new configs are made

* switch ports after new configs are made

* went back to removing to get it to start properly

* seeing if you can use postgres installaition out of the box

* seeing if you can use postgres installaition out of the box

* copy hba file, then stop,start

* testing using port as argument

* working version

* round 2

* updated documentation to reflect changes

* use restart instead of stop/start for postgres

* removed extra lines to ensure consistency with future builds

* reload instead of restart postgres

* restart postgres since we can't currently connect

* Switch to testonly for postgres

* run testonly

* Declare as mongo test since it's handed coded to a mongo uri
2020-04-03 08:53:57 -05:00
Diamond Lewis
985933955f Suppress Test Logs (#6256)
* Suppress Test Logs

This will reduce some of the noise in the tests logs.

* replace deprecated buffer

* remove deprecation warnings

* fix geopoint

* Fix GraphQL

* postgres warnings
2019-12-03 18:21:12 -06:00
Diamond Lewis
f7716f2f87 Schema Cache Improvements (#5612)
* Cache Improvements

* improve tests

* more tests

* clean-up

* test with singlecache

* ensure indexes exists

* remove ALL_KEYS

* Add Insert Test

* enableSingleSchemaCache default true

* Revert "enableSingleSchemaCache default true"

This reverts commit 323e7130fb8f695e3ca44ebf9b3b1d38905353da.

* further optimization

* refactor enforceFieldExists

* coverage improvements

* improve tests

* remove flaky test

* cleanup

* Learned something new
2019-05-24 16:42:27 -05:00
Florent Vilmart
d83a0b6808 Use Prettier JS (#5017)
* Adds prettier

* Run lint before tests
2018-09-01 13:58:06 -04:00
Florent Vilmart
960431b92d Even faster tests (#4868)
* Various improvements in test name / de-duplications

* Reverts to class by class deletion, introduced fast mode that just delete data for mongo

- Speeds up are incredible Executed 1695 of 1713 specs INCOMPLETE (18 PENDING) in 4 mins 19 secs.

* Adds documentation about the deleteEverything
2018-07-03 11:13:08 -04:00
Florent Vilmart
ae1a8226d5 Removes need to use babel-register (#4865)
* Removes need to use babel-register

- Adds watch to watch changes when running the test to regenerate
- Tests are now pure node 8

* Adds timing to helper.js

* Update contribution guide

* Adds inline sourcemaps generation to restore coverage

* nits
2018-07-02 23:30:14 -04:00
Florent Vilmart
585938ca2c improves test performance on mongodb (#4862)
* improves test performance on mongodb

* Removes unused methdos
2018-06-29 17:09:51 -04:00
Florent Vilmart
b754d51e8e chore(package): update jasmine to version 3.0.0 (#4553)
* chore(package): update jasmine to version 3.0.0

Closes #4547

* Fixes failing tests for jasmine 3.0

Starting 3.0, done(something) will fail

* Update tests so they dont leverage var, but let and const

With jasmine 3.0, the randomization engine was making the test fails because of the scope of `var`

* Remove randomizer

* Use same adapter for PG tests, drop table to ensure the tests dont side effect
2018-02-17 09:55:30 -05:00
Paulo Reis
cb8f038ca8 #4338 pg schema upgrade (#4375)
* Method to upgrade schemas in Postgres;

Adds:
  - schemaUpgrade method which check the fields to add and delete from old schema;

* Remove the columns delete in schemaUpgrade method;

* ESLint fix and PostgresStorageAdapter.schemaUpgrade spec test

Adds:
  - Add PostgresStorageAdapter.schemaUpgrade spec tests: creates a table,
    simulates the addition of a new field and checks if its present in the database
Chores:
  - ESLint eol-last fix;

* Add check columns before and after schema upgrade, and remove the unnecessary piece of code

Add:
  - Check the right columns is present before schema upgrade and the new field is not,
    then check if the right columns is present and the new field;
Remove:
  - Piece of code unnecessary because it not need to remove diff columns;

* Optimize the schemaUpgrade method following @vitaly-t instructions, and more tests;

* If the class does not have any columns and needs an upgrade the code would
return without doing so. Fixed this.

Chore:
  - Allows class with no column to be upgraded;
  - Test for class with no columns being upgraded;

* Update PostgresStorageAdapter.js

rewriting method schemaUpgrade

* Update PostgresStorageAdapter.spec.js

* Update PostgresStorageAdapter.spec.js
2018-01-03 03:23:05 +00:00
Florent Vilmart
10631371e6 Introduces flow types for storage (#4349)
* Introduces flow types for storage

* Better typing of QueryOptions

* Adds flow types to SchemaCOntroller,

- runs flow on pre tests
- fixes flow

* Adds ClassLevelPermissions type

* Moves Controller types into a single file

* Changes import styles

* Changes import styles

* fixing method setIndexesWithSchemaFormat (#4454)

Fixing invalid database code in method `setIndexesWithSchemaFormat`:

* It must be a transaction, not a task, as it executes multiple database changes
* It should contain the initial queries inside the transaction, providing the context, not outside it;
* Replaced the code with the ES6 Generator notation
* Removing the use of batch, as the value of the result promise is irrelevant, only success/failure that matters

* nits

* Fixes tests, improves flow typing
2017-12-30 20:44:18 -05:00
CHANG, TZU-YEN
8bf6abfee3 Patch handleShutdown feature (#4361)
* Add handleShutdown Test of MongoStorageAdapter.

* Add `handleShutdown` function in PostgresStorageAdapter, with it's test.

* Replace the _pgp to _client

* Use  `adapter._client.$pool.ending` to ckeck is ending

* Add `handleShutdown()` in ParseServer.spec.js

* Trigger

* Set Server to Default
2017-11-18 15:20:19 -05:00