Salem's Euphoria

Sharing Experience


Leave a comment

Do Not Require Kerberos Pre-Authentication, for users created by Ambari on AD

 

Disclaimer:

Microsoft says that “Disabling Kerberos Pre-Authentication must not be disabled“. They argue that:

Without Kerberos Pre-Authentication a malicious attacker can directly send a dummy request for authentication. The KDC will return an encrypted TGT and the attacker can brute force it offline. Upon checking the KDC logs, nothing will be seen except a single request for a TGT.

I don’t believe that pre-authentication adds any level of security as you can read in this nice article (section 3.1).

If an attacker is able to capture the preauthentication packets and wants to take the identity of a valid user, the attacker will need to perform the procedures that the KDC performs. The attacker will need to use the decryption procedure in the agreed upon encryption type and try running different passwords against the captured data. If it is successful then the attacker has the user’s password. The time required for this procedure is a function of the complexity of the password and the time it takes to decrypt a single password

So, if you decide to do it, you may act only on Your own behalf and on Your sole responsibility 😀 .

Continue reading

Advertisements


Leave a comment

How to disable Kerberos in Ambari?

Ambari-logo-300x141

When you fall in the case where your KDC is down and you can’t reach it anymore, it will impossible for you to disable the Kerberos Security through Ambari GUI. Actually, when you request Ambari to disable the Kerberos security via its GUI, it will try to start your Zookeeper to update the different nodes’ configuration. However, if your KDC (or Windows Active Directory) is unreacheable, you can’t go further.

Broken Kerberos Installation

You may try to manually edit files or clean the Ambari database. If you have no luck with this, try to use the REST API, and here is how.

First, create a json file, where you will write the PUT call body:

disable_krb.json

{
"Clusters": {
"security_type": "NONE"
}
}

 

Then use this body call using a curl resquest:

curl -H “X-Requested-By:ambari” -u admin:admin -i -X PUT -d @./disable=_krb.json http://your.ambari.server:port/api/v1/clusters/YOUR_CLUSTER_NAME

This should invoke the Ambari process “Disable Security”.

2018-04-03_1023

You can find many other details about Kerberos REST management in the groovy KerberosService.


Leave a comment

Ambari – Remove a Host

I hope this will get easier with Ambari next time. The easiest way to remove a host server1.domain.net from an Ambari Cluster (ambari.domain.net) right now is:

1 – Run this curl command to get installed services on this host (admin:admin is the username:password you were using to access your Ambari GUI, cluster_name is the name of your cluster).

curl -u admin:admin -H "X-Requested-By: ambari" -X GET http://ambari.domain.net:8080/api/v1/clusters/cluster_name/hosts/server1.domain.fr/host_components | grep host_components

The result will look like :

"href" : "http://ambari.domain.net:8080/api/v1/clusters/cluster_name/hosts/server1.domain.net/host_components/HBASE_CLIENT",
"href" : "http://ambari.domain.net:8080/api/v1/clusters/cluster_name/hosts/server1.domain.net/host_components/HCAT",
"href" : "http://ambari.domain.net:8080/api/v1/clusters/cluster_name/hosts/server1.domain.net/host_components/HDFS_CLIENT",
"href" : "http://ambari.domain.net:8080/api/v1/clusters/cluster_name/hosts/server1.domain.net/host_components/HIVE_CLIENT",
"href" : "http://ambari.domain.net:8080/api/v1/clusters/cluster_name/hosts/server1.domain.net/host_components/HIVE_METASTORE",
"href" : "http://ambari.domain.net:8080/api/v1/clusters/cluster_name/hosts/server1.domain.net/host_components/HIVE_SERVER",
"href" : "http://ambari.domain.net:8080/api/v1/clusters/cluster_name/hosts/server1.domain.net/host_components/HST_AGENT",
"href" : "http://ambari.domain.net:8080/api/v1/clusters/cluster_name/hosts/server1.domain.net/host_components/INFRA_SOLR_CLIENT",
"href" : "http://ambari.domain.net:8080/api/v1/clusters/cluster_name/hosts/server1.domain.net/host_components/KNOX_GATEWAY",
"href" : "http://ambari.domain.net:8080/api/v1/clusters/cluster_name/hosts/server1.domain.net/host_components/MAPREDUCE2_CLIENT",
...
"href" : "http://ambari.domain.net:8080/api/v1/clusters/bopam_clu/hosts/server1.domain.net/host_components/METRICS_MONITOR",
"href" : "http://ambari.domain.net:8080/api/v1/clusters/bopam_clu/hosts/server1.domain.net/host_components/MYSQL_SERVER",

2 – Delete the listed services by issuing the following curl command for each SERVICE NAME :

curl -u admin:admin -H "X-Requested-By: ambari" -X DELETE http://ambari.domain.net:8080/api/v1/clusters/cluster_name/hosts/server1.domain.net/host_components/SERVICE_NAME

Ex :

curl -u admin:admin -H "X-Requested-By: ambari" -X DELETE http://ambari.domain.net:8080/api/v1/clusters/cluster_name/hosts/server1.domain.net/host_components/HBASE_CLIENT

3 – You can now delete the host:

curl -u admin:admin -H "X-Requested-By: ambari" -X DELETE http://ambari.domain.net:8080/api/v1/clusters/cluster_name/hosts/server1.domain.net

 

4 – Check the rest of the services depending on this host.

We are always still waiting for the Ambari multi-cluster support and the famous multi-everything architecture from HortonWorks team.


Leave a comment

Implicit updateRequestProcessorChain call – Solr

How to concat two fields in schemaless mode with Solr in Cloud mode?

1 – Create a js script file (concat_fields.js) and edit the following code:


function processAdd(cmd) {
doc = cmd.solrDoc;
id = doc.getFieldValue("id");
val1= doc.getField('field1').getValue();
val2 = doc.getField('field2').getValue();
separator = params.get('separator');
doc.setField("field3", val1+separator+val2);
}

Continue reading


Leave a comment

Ambari uninstall scripts

 

If you want to remove an Ambari-through install of HDP components, you will have to it manually.

Actually, Ambari gives you a way to uninstall services and remove hosts. But, this feature assumes that you have completed the components install process successfully. What if the process fails somewhere when installing?

In my case, I collected all the commands from many posts and grouped them into one single script. You can find the script in my github.

By the way, this is a very well written manual on how to setup HDP 2.5.


2 Comments

Setup HBase Indexer (Part 2)

1 – Why would someone use Solr to search on a wide-column database (HBase)?

The power of HBase search (scans) is not filters. All is about the rowkey design. If you want to take full advantage of HBase, you must know all your search queries at the moment of deigning your database. This way, you will put all the “search” intelligence in your rowkeys. But what if you don’t know all your search criteria at the beginning? What if you need to add extra search criterias? Would you create a new “view” of data with another rowkey strategy? What would you do if your client needs to search by “proximity” or a did you mean style?

There is no answer for this question than “it depends”.

 

2 – Why we did not use Ambari for Solr deployment?

It is not integrated offcially, it does not bring any added-value, it adds some more complexity in ambari-agents scripts (must be altered manually for this use case).

Continue reading


Leave a comment

Setup HBase Indexer (Part 1)

Pre-requisites:

The scope of this post does not cover Hadoop/Hbase setup. I asume that you have a running Hbase environment with a Master (HMaster) and two region servers (rs1 and rs2).

I’ll be using the HDP2.5 release from HortonWorks setup on CentOS 7.2.

1 – Setup Solr

Actually, I don’t want Ambari to manage my Solr instance because, we have some specific configurations to add and we won’t alter default ambari-agent’s behaviour.

sudo rpm --import http://public-repo-1.hortonworks.com/HDP-SOLR-2.5-100/repos/centos6/RPM-GPG-KEY/RPM-GPG-KEY-Jenkins
sudo cd /etc/yum.repos.d/
sudo wget http://public-repo-1.hortonworks.com/HDP-SOLR-2.5-100/repos/centos7/hdp-solr.repo
sudo yum install lucidworks-hdpsearch

Continue reading