HOAB

History of a bug

Postgresql pg_upgrade failed : role <XXX> unknown

Rédigé par gorki Aucun commentaire

Problem :

When migrating a database, pg_upgrade failed with “could not connect, role <XXX> is unknown”.

My database was created with another user (postgresql)

Then transfered to another user (let's call it john)

The login authentication was password only until upgrade, where I put local to trust for upgrade.

The new destination database was created by john.

Solution :

The pg_upgrade can not use : 

  • an old database with super role postgres
  • a new database with super role john

The option “-U <username>” is applied to the both, so there is always one which is wrong.

So rename the old database super role to john (thanks to internet).

  1. start the old database, pg_upgrade give the command in the log but !!
  • Be careful ! there is a hidden option “-b” which prevent any modification :) remove it and all is ok
"/home/myuser/postgresql-9.6.2/bin/pg_ctl" -w -D "/home/myuser/database-introscope-9.6" -o "-p 50432  -c listen_addresses='' -c unix_socket_permissions=0700 -c unix_socket_directories='/home/myuser'" start

Connect with : 

/home/myuser/postgresql-9.6.2/bin/psql -p 50432 -h /home/myuser -U postgres -d postgres

Check the super user role name : 

SELECT rolname FROM pg_roles WHERE oid = 10;

Create another super user

CREATE ROLE spiderman SUPERUSER LOGIN PASSWORD 'moreSecurePass';

Quit, connect with spideman, still on postgres database

/home/myuser/postgresql-9.6.2/bin/psql -p 50432 -h /home/myuser -U spideman -d postgres

Rename the original role :

alter role postgres rename to john;

Check the connection :

/home/myuser/postgresql-9.6.2/bin/psql -p 50432 -h /home/myuser -U john -d postgres

Drop spiderman role : 

DROP ROLE spiderman;

I learn to side tricks here : 

  • -h <path> : give the path to the unix socket connection
  • -b option : use for binary upgrade mode on pg_ctl (didn't see the option in the documentation)
  • Owner role has value OID=10

 

 

 

Relocatable postgresql

Rédigé par gorki Aucun commentaire

Problem :

I need a relocatable / portable version of postgresql.

Disclaimer : if possible, use the OS distribution, it will be more up-to-date and sure.

That thing said : 

  • I downloaded the source
  • Compile it on my computer (Debian / Trixie / Sid)
  • Package and run

Fail ! Obviously but here is why : 

  1. Shared library are related to an absolute path
  2. Glibc version on my computer is more recent than target server

Solution :

Two problems here and a few useful commands :

  • ldd <program name>
    • Will give you if the program is static or dynamic
    • If dynamic, the list of libraries and the searched path, example (libpq is a library from postgres)
 libpq.so.5 => /lib/x86_64-linux-gnu/libpq.so.5 (0x00007f29f9786000)
  • readelf -d <program name> : information on execution
0x0000000000000001 (NEEDED)             Shared library : [libpq.so.5]
0x0000000000000001 (NEEDED)             Shared library : [libm.so.6]
0x0000000000000001 (NEEDED)             Shared library : [libc.so.6]
0x000000000000001d (RUNPATH)            Library runpath :[/home/myuser/projects/packaging/postgresql/output/lib]
0x000000000000000c (INIT)               0x6000
0x000000000000000d (FINI)               0x162a0
  • objdump -T <program name> | grep GLIBC : list dependencies on GLIBC

 

Once I had this information : 

  • I compile postgresql on the target system : GLIBC 2.34, 2.33 were not available so postgresql compile without linking it
  • I change the runpath of the executables with these commands :
# extract postgres sources
./configure --prefix=/home/myuser/postgresql/output --without-icu --without-readline --without-zlib --disable-rpath
export LD_RUN_PATH='$ORIGIN/../lib'
make
make install

It generates a postgresql version in  /home/myuser/postgresql/output

The readelf commands returns : 

 0x000000000000000f (RPATH)              Library rpath: [$ORIGIN/../lib]

So I was now able to package it.

 

 

AtomicLong vs Long

Rédigé par gorki Aucun commentaire

Problem :

I searched for an simple example of AtomicLong vs Long problem.

Thanks to Ecosia IA (ChatGPT), here is a simple example

Solution :

Result is : 

  • Counter value (Long): 34938
  • Counter value (AtomicLong): 100000

Here is the code :

package tools;
import java.util.concurrent.atomic.AtomicLong;

public class AtomicLongExample {
    private static Long counter = 0L;
    private static AtomicLong atomicCounter = new AtomicLong(0);

    public static void main(String[] args) {
        // Create and start multiple threads
        Thread[] threads = new Thread[100];
        for (int i = 0; i < threads.length; i++) {
            threads[i] = new Thread(new CounterRunnable());
            threads[i].start();
        }

        // Wait for all threads to finish
        for (Thread thread : threads) {
            try {
                thread.join();
            } catch (InterruptedException e) {
                e.printStackTrace();
            }
        }

        System.out.println("Counter value (Long): " + counter);
        System.out.println("Counter value (AtomicLong): " + atomicCounter.get());
    }

    static class CounterRunnable implements Runnable {
        @Override
        public void run() {
            // Increment the counter using Long variable within a loop
            // This may lead to race conditions and incorrect results
            for (int i = 0; i < 1000; i++) {
                counter++;
            }

            // Increment the counter using AtomicLong within a loop
            // This ensures atomicity and thread-safety
            for (int i = 0; i < 1000; i++) {
                atomicCounter.incrementAndGet();
            }
        }
    }
}

SpringBoot OSGI and templated OSGI modules with external constrant Jetty

Rédigé par gorki Aucun commentaire

Problem :

A very specific and particular problem.

I want to deploy  a SpringBoot application, in an OSGI environment, in a external Jetty server…

So I found solution do deploy Springboot in OSGI : here or here. Very good by the wat.

And also to deploy Springboot as war for an external Tomcat…

Too easy, my external Jetty server was rejecting Servlet 3.0 application startup… (hard coding for : 

context.setAttribute("org.eclipse.jetty.containerInitializers", initializers);

Yes, it's better, starting a 1,25 days of try and success !

Solution :

My first web application was deployed with Spring Dispatcher Servlet with a BundleActivator

My second web application was deployed with Spring DispatcherServlet AND a SpringBootApplication intialized with the ressource loader of the BundleActivator (as in the OSGI example) : component-scan is working ! 

My third web application was switching to an initialization Servlet 3.0 with an extension of ServletContainerInitializer, thanks to harded jetty configuration, it's not working

My third and a half remove all web.xml content to keep only a context listener as my Jetty was doing nothing : 

<?xml version="1.0" encoding="ISO-8859-1"?>
<web-app xmlns="http://java.sun.com/xml/ns/javaee"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://java.sun.com/xml/ns/javaee http://java.sun.com/xml/ns/javaee/web-app_3_0.xsd"
         version="3.0">

    <!-- ServletContainerInitializer can not be used here -->
    <!-- XXXXXXX is putting in hard : context.setAttribute("org.eclipse.jetty.containerInitializers with only Jasper -->

    <!-- At the end, it's possible to load SpringBoot with this context listener -->
    <listener>
        <listener-class>
            com.hexagon.introscope.extension.MyContextLoaderListener
        </listener-class>
    </listener>

</web-app>

My fourth web application was trying to load the Springboot application file as a YAML file (why so simple !), resolved with a “hack” :

  • The ServletContext has a way to store the configuration directory, during the initialization, I saved it somewhere : HpaContextLoaderListener.getConfigDirectory() 

The listener loader : 

package com.xxxxx.extension;

import org.eclipse.jetty.webapp.WebAppContext;
import org.springframework.web.context.WebApplicationContext;

import javax.servlet.ServletContext;
import javax.servlet.ServletContextEvent;
import javax.servlet.ServletContextListener;
import javax.servlet.ServletException;
import java.io.File;
import java.nio.file.Path;

public class MyContextLoaderListener implements ServletContextListener {

    private static File configDirectory;

    public static File getConfigDirectory() {
        return configDirectory;
    }

    public void contextInitialized(ServletContextEvent event) {
        ServletContext servletContext = event.getServletContext();
        MySpringBootHexagonActivator app = new MySpringBootHexagonActivator();

        try {
            configDirectory = servletContext.getAttribute("introscope.em"));
            
            InstantiateHelper.getValueByMethod(servletContext, "setExtendedListenerTypes", new Class[]{boolean.class}, true);

            app.onStartup(event.getServletContext());
        } catch (ServletException e) {
            throw new RuntimeException(e);
        }
    }

    public void contextDestroyed(ServletContextEvent event) {
    }
}

 

@SpringBootApplication
public class MySpringBootHexagonActivator extends SpringBootServletInitializer {

    protected SpringApplicationBuilder createSpringApplicationBuilder() {
        SpringApplicationBuilder builder = super.createSpringApplicationBuilder();
        Map<String, Object> properties = new HashMap<>();
        Path propertyFile = Path.of(MyContextLoaderListener.getConfigDirectory().getPath(), "application-extension.yml");
        properties.put("spring.config.location", propertyFile.toString());
        builder.properties(properties);
        builder.resourceLoader(HpaHexagonActivator.getResourceResolver());

        return builder;
    }
}

The HpaHexagonActivator.getResourceResolver() is the trick from SpringBoot OSGI example to load classes from Bundle classpath. I store the ressources resolver in a static variable that I can access from everywhere.

A word on that one : 

InstantiateHelper.getValueByMethod(servletContext, "setExtendedListenerTypes", new Class[]{boolean.class}, true);

I'm in OSGI, Jetty controls the listener class, and Springboot is unknown. You can authorized additional listeners but in my case, Jetty is not exported by the other OSGI package, not able to call it simply so… reflection : InstantiateHelper is basically a helper class to call methods by reflection.

 

At the end : 

  1. Springboot starts
  2. Springboot scans package offered by BundleActivator ressource locator
  3. Springboot can use an external configuration file, in YAML
  4. Springboot starts REST & Database
  5. It was hard.

Of course, lot of tries between these big steps !

The goal is to extend a 3rd party product without having source code.

Angular @Input array was not updated.

Rédigé par gorki Aucun commentaire

Problem :

Here again, a common problem, the @Input attribute of the child element was not updated.

Good to know : 

  • ngOnChanges is called when the @Input attribute is already updated
  • common problem : the @Input parameter is an array and its reference is not modified, i.e, the array is the same even if it has different elements or new elements

No, in my case, the symptoms was : 

  1. ngOnChanges input is launch
  2. SimpleChanges event contains the data in “currentValue
  3. but array is displayed as [] even with values.

Yeah, tricky. I understand that array may be an object instead of a real array… but why. 

Solution :

Part of the problem was that the value sent to @Input was an array, but not initialized.

And a small function used to have unique value : 

  private pushIfNotExist<T>(array:T[], element:T) {
    if (array.indexOf(element) === -1) {
      array.push(element);
    }
  };

and I call it like that : 

pushIfNotExist(myArray, randomValue)

And with an empty javascript variable, well : no problem, it works, in a given way.

Adding : 

myArray = []

just before, everything works as a charm.

I lose time here, because I had a doubt on where was the issue : display on browser console, workflow for triggering ngOnChange, …

 

 

Lire la suite de Angular @Input array was not updated.

Fil RSS des articles