toc
| Table of Contents |
|---|
Key Points
- provides REST api to NoSQL documents
- supports basic aggregation
- has Fauxton admin browser for Web queries using NoSQL without coding
- does not support JDBC access from other tools directly for easy reporting
- see CAMEL for some data integration options with Java, XML
- CouchDB v4x will use FoundationDB key-value store as the storage engine
- Mango query in Fauxton client is similar to Mongo Query syntax
- Berlin couchdb meetup site - jm0 for jem1
References
...
https://www.geeksforgeeks.org/top-10-open-source-nosql-databases-in-2020/
Apache CouchDB is an open-source project and a single node database that allows you to easily store your data and access it when you need it. Couch DB can also scale up for more demanding projects into a cluster of nodes with multiple servers. It supports the HTTP protocol along with the JSON data format and also integrates with HTTP proxy servers. Apache CouchDB is designed for reliability with a crash-resistant structure that supports “Offline First” applications and a system that saves data redundantly so that it is never lost and available in a state of emergency.
CouchDb Setup
https://docs.couchdb.org/en/stable/install/index.html
...
Fauxton setup
Go to couchdb.apache.org, and click download
Learn how to install and setup CouchDB from here, then go to http://127.0.0.1:5984/_utils
...
The list of conditional arguments:
(In)equality operators
...
operators
- "$lt" - any JSON
- "$lte" - any JSON
- "$eq" - any JSON
- "$ne" - any JSON
- "$gte" - any JSON
- "$lte$gt" - any JSON
Object related operators
...
Object related operators
- "$exists" - boolean, check whether the field exists or not regardless of its value
- "$type" - string, check the document field's type
Array related operators
- "$in" - array of JSON values, the document field must exist in the list provided
- "$nin" - array of JSON values, the document field must not exist in the list provided
- "$size" - integer, special condition to match the length of an array field in a document. Non-array fields cannot match this condition.
Misc related operators
- "$mod" - [Divisor, Remainder], where Divisor and Remainder are both positive integers (ie, greater than 0). Matches documents where (field % Divisor == Remainder) is true. This is false for any non-integer field
- "$regex" - string, a regular expression pattern to match against the document field. Only matches when the field is a string value and matches the supplied matches
Sort
The sort syntax is a basic array of field name and direction pairs. It looks like such:
[{field1: dir1} | ...]
Where field1 can be any field (dotted notation is available for sub-document fields) and dir1 can be "asc" or "desc".
Fields
Unlike MongoDB only the fields specified are included, there is no automatic inclusion of the "_id" or other metadata fields when a field list is included.
A trivial example:
["foo", "bar", "baz"]
More Mango Query Examples - regex, lists, starting date
...
CouchDB migration to FoundationDB
https://forums.foundationdb.org/t/update-couchdb-4-0-on-foundationdb/1690
FoundationDB JDBC driver notes
https://javalibs.com/artifact/com.foundationdb/fdb-sql-layer-jdbc
IDEA - implement JDBC driver for Mango client to connect BI tools
support defined JDBC type 4 client interfaces ..
Driver
Connection
Request
Response
Error protocols
PSCouchDB, a complete and Object Oriented cli for CouchDB
...
Hello everybody,
I wanted to announce the new version of PSCouchDB, a complete and Object Oriented cli for CouchDB, written in powershell (running on any operating system).
Find the release notes here: https://github.com/MatteoGuadrini/PSCouchDB/releases/tag/2.5.0
Other useful links
Installation: https://github.com/MatteoGuadrini/PSCouchDB#installation-and-simple-usage
Full docs: https://pscouchdb.readthedocs.io/en/latest/
PowershellGallery: https://www.powershellgallery.com/packages/PSCouchDB/2.5.0
Site: https://matteoguadrini.github.io/PSCouchDB
Thanks to everyone!
Matteo Guadrini
Potential Value Opportunities
Potential Challenges
Engineering CouchDB Networks
...
Issue - Using a JDBC driver to connect ot CouchDB
https://lists.apache.org/list.html?user@couchdb.apache.org
Issue - CouchDB connection when server certs expired
...
- "$exists" - boolean, check whether the field exists or not regardless of its value
- "$type" - string, check the document field's type
Array related operators
- "$in" - array of JSON values, the document field must exist in the list provided
- "$nin" - array of JSON values, the document field must not exist in the list provided
- "$size" - integer, special condition to match the length of an array field in a document. Non-array fields cannot match this condition.
Misc related operators
- "$mod" - [Divisor, Remainder], where Divisor and Remainder are both positive integers (ie, greater than 0). Matches documents where (field % Divisor == Remainder) is true. This is false for any non-integer field
- "$regex" - string, a regular expression pattern to match against the document field. Only matches when the field is a string value and matches the supplied matches
Sort
The sort syntax is a basic array of field name and direction pairs. It looks like such:
[{field1: dir1} | ...]
Where field1 can be any field (dotted notation is available for sub-document fields) and dir1 can be "asc" or "desc".
Fields
Unlike MongoDB only the fields specified are included, there is no automatic inclusion of the "_id" or other metadata fields when a field list is included.
A trivial example:
["foo", "bar", "baz"]
More Mango Query Examples - regex, lists, starting date
{
"selector": {
"offerKWH": {
"$gte": 80 },
"startTS": {
"$gte": "2021-01-27"}
} }
{
"selector": {
"objectType": {
"$in": [
"device",
"bid",
"offer",
"trade"
] },
"Timestamp": {
"$gte": "2021-02-22"
} } }
{
"selector": {
"name": {
"$regex": "^Test acc"
} } }
CouchDB migration to FoundationDB
https://forums.foundationdb.org/t/update-couchdb-4-0-on-foundationdb/1690
FoundationDB JDBC driver notes
https://javalibs.com/artifact/com.foundationdb/fdb-sql-layer-jdbc
IDEA - implement JDBC driver for Mango client to connect BI tools
support defined JDBC type 4 client interfaces ..
Driver
Connection
Request
Response
Error protocols
PSCouchDB, a complete and Object Oriented cli for CouchDB
Matteo Guadrini <matteo.guadrini@hotmail.it>
Mon, Jun 27 at 4:39 AM
Hello everybody,
I wanted to announce the new version of PSCouchDB, a complete and Object Oriented cli for CouchDB, written in powershell (running on any operating system).
Find the release notes here: https://github.com/MatteoGuadrini/PSCouchDB/releases/tag/2.5.0
Other useful links
Installation: https://github.com/MatteoGuadrini/PSCouchDB#installation-and-simple-usage
Full docs: https://pscouchdb.readthedocs.io/en/latest/
PowershellGallery: https://www.powershellgallery.com/packages/PSCouchDB/2.5.0
Site: https://matteoguadrini.github.io/PSCouchDB
Thanks to everyone!
Matteo Guadrini
CouchDB JWT authentication and Identity Provider’s JWKS URLs ( beta feature )
Potential Value Opportunities
Potential Challenges
Engineering CouchDB Networks
mesh-like architecture which uses CouchDB replication protocol to
distribute/aggregate data is perfectly viable. As well as other members of
the ML I also have several projects employing this approach. And like
others I also can not disclose any details, even qty of nodes involved; you
better be ready to see this attitude for any mesh-related project, because
they often involve processing sensitive personal, industry or
infrastructure data in vast amounts.
However, I have a lot of observations, main are:
- whenever possible use CouchDB, not Pouch, because failure rate of
Pouch instances on whatever platform is much higher overall
- CouchDB is able to run for years without getting down, Pouch is not so
sturdy, there exist a bunch of ways to knock down Pouch node with a single
request
- on mobile devices you better avoid constantly growing DBs, you may
loose data and DB integrity eventually
- a single doc rejected by Couch during upstream replication from Pouch
can block entire process of replication, you can easily get the situation
when sync just can’t restart – so carefully check what you store in Pouch,
especially attachments size
- get ready to spend a lot of time debugging Pouch—Couch routes having
enormous latencies (like sync through satellites or 14400 GPRS connections)
- mesh CouchDB networks tend to have very fast growing logs, reason is
the number of repeated connections from different peers, so take care of it
- do not distribute mesh topology across nodes using CouchDB
replication, this approach looks inviting but brings a lot of risks.
Best regards.
ermouth
Issue - Using a JDBC driver to connect ot CouchDB
https://lists.apache.org/list.html?user@couchdb.apache.org
Issue - CouchDB connection when server certs expired
https://stackoverflow.com/questions/20082893/unable-to-verify-leaf-signature
setting NODE_TLS_REJECT_UNAUTHORIZED might fix your problem.
If you don't have access to the code, you can set a shell variable and
restart Node:
export NODE_TLS_REJECT_UNAUTHORIZED=0
CAUTION: This should only be used as a TEMPORARY fix until you can get your
certs updated. It opens you up to attacks.
Cheers,
- Bill
On Mon, Oct 18, 2021 at 3:44 PM Rick Jarvis <rick@magicmail.mooo.com> wrote:
> Can anyone tell me (urgently lol) how to ignore invalid certificates with
> nano?
>
> My backups haven’t been running and I’m desperate.
>
> I’ll then need to figure out why, despite having updated all the routes
> etc (due to R3 expiry) it’s still not working!
Candidate Solutions
CouchDb v3.4 updates
CouchDB 3.4.0 and 3.4.1
full text search for couchdb
add embedded QuickJS engine
release 3.4.0 problems
avoid breaking changes and support CouchDB version rollback now in 3.4.1
new hashing methods
QuickJS
Spider Monkey from Firefox still the default js engine in CouchDB
or change config in design doc for flex
test engine
Compare CouchDB v4x with FoundationDB to LevelDB
...
https://github.com/couchbaselabs/couchbase-jdbc-driver
SQL Select ONLY - no CRUD
This project contains the source code for the Couchbase JDBC Driver which supports the Analytics Service (not Query!). Its main purpose is to provide the low-level glue to facilitate integration with high level BI-Tools like Tableau.
Couchbase connection via Apache Drill
...
Programming language bindingsFoundationDB supports language bindings for Python, Go, Ruby, Node.js, Java, PHP, and C, all of which are made available with the product.[13]
Design limitations[edit]
The design of FoundationDB results in several limitations:
...
https://apple.github.io/foundationdb/performance.html
FoundationDB uses commodity hardware to provide high throughputs and low latencies to your application at a variety of scales
FoundationDB scales linearly with the number of cores in a cluster over a wide range of sizes.
Here, a cluster of commodity hardware scales to 8.2 million operations/sec doing a 90% read and 10% write workload with 16 byte keys and values between 8 and 100 bytes.
...