You're printing the address of the pointer, not the contents of the pointer.

Say you have the memory map of:

100 20       test1 low byte
101 0        test1 high byte
102 100      ptr1 low byte
103 0        ptr1 high byte

test is located at address 100 and takes two bytes. ptr1 is located at 102 and takes two bytes. ptr1 contains the address of test (100) and test1 contains the value 20.

The different possible combinations of prints are:

Serial.println(test1);  => 20 (contents of test1)
Serial.println(&test1); => 100 (address of test1)
Serial.println(ptr1);   => 100 (contents of ptr1 = address of test1)
Serial.println(&ptr1);  => 102 (address of ptr1)
Serial.println(*ptr1);  => 20 (contents of address pointed to by ptr1).

You're printing the address of the pointer, not the contents of the pointer.

Say you have the memory map of:

100 20       test1 low byte
101 0        test1 high byte
102 100      ptr1 low byte
103 0        ptr1 high byte

test is located at address 100 and takes two bytes. ptr1 is located at 102 and takes two bytes. ptr1 contains the address of test (100) and test1 contains the value 20.

The different possible combinations of prints are:

Serial.println(test1);  => 20 (contents of test1)
Serial.println(&test1); => 100 (address of test1)
Serial.println(ptr1);   => 100 (contents of ptr1 = address of test1)
Serial.println(&ptr1);  => 102 (address of ptr1)
Serial.println(*ptr1);  => 20 (contents of address pointed to by ptr1).

(note: some of those will require a cast in order for the C++ to select the correct println() overload function.)