Friday, June 2, 2023

Vector store pricings

 

candidates

https://js.langchain.com/docs/modules/indexes/vector_stores/integrations/


typesense cloud $14.40

https://cloud.typesense.org/pricing



https://qdrant.tech/pricing/





https://www.pinecone.io/pricing/

Free plan is not available for now.




https://weaviate.io/pricing

Standard

from

25 /mo
  • $0.050 per 1M vector dimensions stored
    or queried per month




  • supabase

  • https://supabase.com/blog/openai-embeddings-postgres-vector
  • vercel postgres

  • https://vercel.com/templates/next.js/postgres-pgvector




  • Monday, May 1, 2023

    scrapy CrawlSpider instead of wget -r

     

    I needed to download all pages from a wix-made website.

    `wget -r` didn't work.

    httrack , lynx didn't work, either.

    https://askubuntu.com/questions/391622/download-a-whole-website-with-wget-or-other-including-all-its-downloadable-con


    I could download websites with CrawlSpider and FollowLink of https://github.com/scrapy/scrapy

    https://www.youtube.com/watch?v=o1g8prnkuiQ


    I'll use playwrite later, too. (instead of selenium)

    https://scrapeops.io/python-scrapy-playbook/scrapy-playwright/


    I found some candidates

    https://github.com/crawlab-team/crawlab


    Sunday, April 30, 2023

    Using async in inline Editor of GCP Dialogflow. But, timeout too short.

     


    Using async in inline Editor of GCP Dialogflow 

    /* experimental: [asyncawait] */

    https://stackoverflow.com/questions/42637630/does-jshint-support-async-await/48772304#48772304


    to know queryText from the user

    https://stackoverflow.com/questions/60408929/how-do-you-read-query-the-response-body-in-api-v2-of-dialogflow-fulfillment


    function handleRequest(request, response) { const queryText = request.body.queryResult.queryText; // use the queryText variable to process the user's input }


    sample code (without async)

    https://github.com/priyankavergadia/AppointmentScheduler-GoogleCalendar/blob/master/index.js


    Dialogflow CX timeout only 30 sec

    Dialogflow ES timeout only 15 sec

    https://groups.google.com/g/dialogflow-cx-edition-users/c/jajSEPqhYZE


    Following code didn't work.

      async function answerQuestion(agent) {

        const response = await axios.post(

    ...    {

          message: JSON.stringify(request.body.queryResult.queryText)

        },

        {

          headers: { 'Content-Type': 'application/json' },

          timeout: 2 * 60 * 1000

        }

        );

        agent.setContext({ name: "Question", result: response });

        agent.add(`${response.data.data.answer} \n\n ${response.data.data.sources}`);

      }


    Wednesday, June 30, 2021

    mysql 5.7 mac app my.cnf

     Mysql 5.7 from mysql 

    $ /usr/local/mysql-5.7.31-macos10.14-x86_64/bin/mysql --help --verbose | grep my.cnf

                          order of preference, my.cnf, $MYSQL_TCP_PORT,

    /etc/my.cnf /etc/mysql/my.cnf /usr/local/mysql/etc/my.cnf ~/.my.cnf


    It works when I created /etc/my.cnf

    $ cat /etc/my.cnf

    [mysqld]

    default_time_zone='-08:00'


    Homebrew mysql

    $ mysql --help --verbose | grep my.cnf

                          order of preference, my.cnf, $MYSQL_TCP_PORT,

    /etc/my.cnf /etc/mysql/my.cnf /usr/local/etc/my.cnf ~/.my.cnf


    /usr/local/etc/my.cnf exists


    Thursday, November 5, 2020

    python json serialization dacite

     

     Handle nested Dict

    https://github.com/konradhalas/dacite


    Tuple issue

    Tuple --> json list --> python list

    from_dict(A, {'x': x}, config=Config(check_types=False))