Sunday, December 24, 2023

Rearrange pages from front and back pdf files which are scanned from two-sided papers

 


  1. Merge 0001.pdf and 0002.pdf into one.
  2. Make a list of indexes. Interleave two indexes. First one is 0 to 19. Second one is reversed as 39 to 20.
  3. Rearrange the merged.pdf as the ordered_indexes list.


I scanned both-sided papers into two pdf files, front pages and reversed back pages. Merge and rearrange pages from two pdf files into one pdf file.


script methods

  1. https://unix.stackexchange.com/questions/53295/how-to-merge-2-pdf-files-with-interleaving-pages-order

tools

  1. https://pdfsam.org/pdfsam-basic/


RearrangePdf-PdfFilePath_Output-PdfFilePath_OrdersList_Input.py


In [4]: from PyPDF2 import PdfFileMerger



In [6]: merger = PdfFileMerger()


In [11]: pdfs = ['/Users/.../Downloads/0001.pdf', '/Users/.../Downloads/0002.pdf']


In [12]: for f in pdfs: merger.append(f)


In [13]: merger.write("a.pdf")


In [14]: merger.close()


In [15]: from PyPDF2 import PdfFileReader, PdfFileWriter


In [16]: reader = PdfFileReader('a.pdf')


In [35]: f_page_idxes = range(20) ; b_page_idxes = reversed(range(20, 40)) ; fb_page_idxes = zip(f_page_idxes, b_page_idxes)


In [41]: ordered_idxes = list(itertools.chain(*fb_page_idxes))


In [42]: writer = PdfFileWriter()


In [43]: for p in ordered_idxes: writer.addPage(reader.getPage(p))


In [45]: with open("arranged_pdf.pdf", "wb") as f: writer.write(f)


Thursday, November 16, 2023

Custom GPT chatbots for websites

  Open AI announced GPTs. GPT store, GPT4 turbo at Open AI dev day. 

We can build custom GPT chatbot for websites from https://docenty.ai/ .

Sunday, June 11, 2023

Vercel database Prisma setup

 


PrismaClientKnownRequestError:
Invalid `prisma.workspace.findMany()` invocation:
The table `public.workspaces` does not exist in the current database.
at pn.handleRequestError (/vercel/path0/node_modules/@prisma/client/runtime/library.js:176:6477)
at pn.handleAndLogRequestError


npx prisma migrate dev

% npx prisma migrate dev

Environment variables loaded from .env

Prisma schema loaded from prisma/schema.prisma

Datasource "db": PostgreSQL database "verceldb", schema "public" at "ep-delicate-dust-021378.us-east-1.postgres.vercel-storage.com:5432"


Error: P3014


% npx prisma migrate resolve --applied 20220223015914_init_database

Environment variables loaded from .env

Prisma schema loaded from prisma/schema.prisma

Datasource "db": PostgreSQL database "verceldb", schema "public" at "ep-delicate-dust-021378.us-east-1.postgres.vercel-storage.com:5432"

Migration 20220223015914_init_database marked as applied.

% npx prisma migrate resolve --applied 20220508151141_add_domain_verification_columns

Environment variables loaded from .env

Prisma schema loaded from prisma/schema.prisma

Datasource "db": PostgreSQL database "verceldb", schema "public" at "ep-delicate-dust-021378.us-east-1.postgres.vercel-storage.com:5432"

Migration 20220508151141_add_domain_verification_columns marked as applied.



Prisma Migrate could not create the shadow database. Please make sure the database user has permission to create databases. Read more about the shadow database (and workarounds) at https://pris.ly/d/migrate-shadow


Original error:

db error: ERROR: permission denied to create database

   0: schema_core::state::DevDiagnostic

             at schema-engine/core/src/state.rs:266



Put env vars to vercel except VERCEL_ prefixed env vars

The value is not a valid System Environment name


Sunday, April 30, 2023

Using async in inline Editor of GCP Dialogflow. But, timeout too short.

 


Using async in inline Editor of GCP Dialogflow 

/* experimental: [asyncawait] */

https://stackoverflow.com/questions/42637630/does-jshint-support-async-await/48772304#48772304


to know queryText from the user

https://stackoverflow.com/questions/60408929/how-do-you-read-query-the-response-body-in-api-v2-of-dialogflow-fulfillment


function handleRequest(request, response) { const queryText = request.body.queryResult.queryText; // use the queryText variable to process the user's input }


sample code (without async)

https://github.com/priyankavergadia/AppointmentScheduler-GoogleCalendar/blob/master/index.js


Dialogflow CX timeout only 30 sec

Dialogflow ES timeout only 15 sec

https://groups.google.com/g/dialogflow-cx-edition-users/c/jajSEPqhYZE


Following code didn't work.

  async function answerQuestion(agent) {

    const response = await axios.post(

...    {

      message: JSON.stringify(request.body.queryResult.queryText)

    },

    {

      headers: { 'Content-Type': 'application/json' },

      timeout: 2 * 60 * 1000

    }

    );

    agent.setContext({ name: "Question", result: response });

    agent.add(`${response.data.data.answer} \n\n ${response.data.data.sources}`);

  }


Wednesday, June 30, 2021

mysql 5.7 mac app my.cnf

 Mysql 5.7 from mysql 

$ /usr/local/mysql-5.7.31-macos10.14-x86_64/bin/mysql --help --verbose | grep my.cnf

                      order of preference, my.cnf, $MYSQL_TCP_PORT,

/etc/my.cnf /etc/mysql/my.cnf /usr/local/mysql/etc/my.cnf ~/.my.cnf


It works when I created /etc/my.cnf

$ cat /etc/my.cnf

[mysqld]

default_time_zone='-08:00'


Homebrew mysql

$ mysql --help --verbose | grep my.cnf

                      order of preference, my.cnf, $MYSQL_TCP_PORT,

/etc/my.cnf /etc/mysql/my.cnf /usr/local/etc/my.cnf ~/.my.cnf


/usr/local/etc/my.cnf exists


Thursday, November 5, 2020

python json serialization dacite

 

 Handle nested Dict

https://github.com/konradhalas/dacite


Tuple issue

Tuple --> json list --> python list

from_dict(A, {'x': x}, config=Config(check_types=False))


Friday, April 4, 2014

Sending touch event to the android device by using monkeyrunner



Using the monkeyrunner, you can control the device with connected on a pc.
It is contained in the android sdk from google.

http://developer.android.com/tools/help/monkeyrunner_concepts.html


Preparation

  1. Install android sdk (recent version, maybe?)
  2. Turn on the debugging option of the device

  3. If DOWN_AND_UP is not working well,
    Settings >  Pointer speed > Lower the speed


Write following python for the first try.

# Imports the monkeyrunner modules used by this program
from com.android.monkeyrunner import MonkeyRunner, MonkeyDevice
# Connects to the current device, returning a MonkeyDevice object
device = MonkeyRunner.waitForConnection()
# Presses the Menu button
device.press('KEYCODE_MENU', MonkeyDevice.DOWN_AND_UP)
# Touches (x, y) position
device.touch(x, y, MonkeyDevice.DOWN_AND_UP)

Connect your device to your pc, confirm it is well being debugging.
Type the following script.
$ adb devices

If it is well connected, it will reply device's some string.

To confirm that the touch is working well, I suggest you run some paint app first such as Sketchbook express, Evernote skitch.

With the device connected to the pc, run monkeyrunner with your python script.

<android sdk directory>/tools/monkeyrunner <your script>.py

(x, y) position is dependent on your device's resolution (pixel coordination).
(500, 500) is ok, I think.

If you want to know exact position of the device, turn some settings on.

  1. Developer options > Input > Show touches CHECK
  2. Developer options > Input > Pointer location CHECK
With this setting, you can check the exact coordinate of the location when you touch somewhere.
Location will be shown on the top of the screen.














Sunday, March 9, 2014

Parse.com android push notifications


It seems to work for sending simple pushes to both android, ios devices.
Installation special object seems to be used to send a certain segment of devices.

https://parse.com/tutorials/android-push-notifications

The Parse library provides push notifications by using Google Cloud Messaging (GCM) if possible. On devices that do not support GCM (such as the Amazon Kindle Fire), Parse will use a background service that maintains a persistent connection to the Parse Cloud to deliver pushes. This allows Parse Push to work on all devices running Android 2.2 or higher.


Some push features are available only for pro users.


Global Push opens metrics Visualize how often your app is opened because of a push notification you sent.
Advanced Push Targeting Ability to perform advanced targeting in the Parse Push console.
Push Scheduling Ability to schedule pushes in the Parse Push console.


Thursday, February 6, 2014

multithreading MessagePack usage without java PermGen memory error


If you use new MessagePack() too many time, perm gem memory error occurs soon.

MessagePack msgpack = new MessagePack();
byte[] bytes = msgpack.write(object);

You can test this with
-XX:+TraceClassLoading -XX:+TraceClassUnloading
VM parameters or JConsole.

[Loaded learningtest.msgpack.MessagePackTest$MyMessage_$$_Template_1527741453_0 from __JVM_DefineClass__]
MyMessage [name=msgpack, version=0.6]
[Loaded learningtest.msgpack.MessagePackTest$MyMessage_$$_Template_831556519_0 from __JVM_DefineClass__]
MyMessage [name=msgpack, version=0.6]
[Loaded learningtest.msgpack.MessagePackTest$MyMessage_$$_Template_1872305359_0 from __JVM_DefineClass__]
MyMessage [name=msgpack, version=0.6]

MessagePack.write()
TemplateRegistry.lookup()
TemplateRegistry.lookupAfterBuilding()
TemplateRegistry.buildAndRegister()
AbstractTemplateBuilder.buildTemplate()
DefaultBuildContext.buildTemplate()
BuildContext.build()
BuildContext.createClass()
CtClass.toClass()


But, writing with the MessagePack instance is not thread-safe.
msgpack.setOutputStream(outputStream);
msgpack.write(object);

Insteads, you can should the packer and the unpacker from a singleton MessagePack instance

MessagePack msgpack = new MessagePack(); // singleton
Packer packer = msgpack.createPacker(outputStream); // createPacker every time
packer.write(object);

for (int i = 0; i < 100; i++) {
final int iTask = i;
tasks.add(new Thread() {

@Override
public void run() {
try {
System.out.println(iTask + " start ");
File file = new File("eraseme" + iTask);
FileOutputStream outputStream = null;

outputStream = new FileOutputStream(file);
SomeClass info = new SomeClass();
try {
packer.write(info);
} catch (IOException e) {
System.out.println(iTask + " write failed " + e.getMessage());
assertTrue(false);
}
outputStream.close();
FileInputStream inputStream = new FileInputStream(file);
Unpacker unpacker = msgpack.createUnpacker(inputStream);
// Deserialize directly using a template
SomeClass reply = null;
try {
reply = (ReplyMessage.ImageInfoMessage) unpacker.read(SomeClass.class);
} catch (IOException e) {
System.out.println(iTask + " read failed " + e.getMessage());
assertTrue(false);
}
System.out.println(iTask + " finished " + reply + " " + file.getAbsolutePath());
} catch (Exception exc) {
System.out.println("exc " + exc.getMessage());
assertTrue(false);
}
};
});
}
for (Thread task : tasks) {
task.start();
}
for (Thread task : tasks) {
task.join();
}

assertTrue(true);


Or, we should use pool of MessagePack instances.
I suggest the apache commons pool.
http://commons.apache.org/proper/commons-pool/



Friday, December 20, 2013

To stop animator determitively, nineoldandroid's Animator.end() is not enough.




Requirements

  1. Animators sometimes should be stopped onPause
  2. The status of properties that animators have been changing should be known to us (activity(?)). It should be determinable.


I wanted to pause animators. And, I used the nineoldandroids for the back compatibility.

If you cancel the animator, you may not be sure that the properties of View are changed for which amount.
This often yields various bugs.

voidcancel()
Cancels the animation.

So, I wanted to

voidend()
Ends the animation.Note that ending a AnimatorSet also ends all of the animations that it is responsible for.


I tried calling animator.end() by hand, but, property value callback was not called
even though nineoldandroids has the mechanism to call child animators of AnimatorSet.

  334
             if (.size() > 0) {
335
                 for (Node node : ) {
336
                     node.animation.end();
337
                 }
338
             }


I'm not sure it's not working for both Android Animator and Animation,

Anyway, I used an workaround of setDuration(0).
If I call animator.setDuration(0), it will be finished soon. But, I'm not sure that it will be finished on the same (UI) thread in the same call stack.

You should know the original duration value since you should roll back the duration after the animator ended.

But, getDuration may not be working... ( for the nineoldandroid only?)
Durations should be set on the AnimatorSet (root animator).
Otherwise, animatorSet.getDuration() returns -1.
And, objectAnimator's duration should be set, too, to support this animation on android 2.*.
<set xmlns:android="http://schemas.android.com/apk/res/android"     android:duration="200" >
    <objectAnimator
        android:duration="200"

longgetDuration()
Gets the length of each of the child animations of this AnimatorSet.
abstract AnimatorsetDuration(long duration)
Sets the duration of the animation.


Maybe, you should roll back the duration value of the original animator even though you a cloned animator.
I don't know why.

AnimatorSetclone()
Creates and returns a copy of this Object.






android Volley can't handle redirect responses for twipic images even though we do setFollowRedirect.


I think android volley can't handle redirect responses by default (?)
But, we make URLConnection, which volley uses on android 2.3 (API 9) or higher(?), handles redirects with setFollowRedirects.
You can override HurlStack.createConnection() or simply call HttpURLConnection.setFollowRedirects().

static voidsetFollowRedirects(boolean auto)
Sets the flag of whether this connection will follow redirects returned by the remote server.
voidsetInstanceFollowRedirects(boolean followRedirects)
Sets whether this connection follows redirects.

https://android.googlesource.com/platform/frameworks/volley/+/master/src/com/android/volley/toolbox/HurlStack.java
  1.     protected HttpURLConnection createConnection(URL url) throws IOException {
  2.         return (HttpURLConnection) url.openConnection();
  3.     }


But, it still can't handle twipic images.
The reason why was that setFollowRedirects doesn't allow HTTP <--> HTTP redirects.

http://developer.android.com/reference/java/net/HttpURLConnection.html

Response Handling

HttpURLConnection will follow up to five HTTP redirects. It will follow redirects from one origin server to another. This implementation doesn't follow redirects from HTTPS to HTTP or vice versa.


If you still want to fix it, modify the source code of the volley, please.

http://stackoverflow.com/questions/17481964/android-volley-to-handle-redirect
http://stackoverflow.com/questions/19787999/unexpected-response-code-302-for-volley-image-request